-- Logs begin at Fri 2022-05-27 02:22:04 UTC, end at Fri 2022-05-27 04:00:09 UTC. -- May 27 03:13:36.382929 dv-cinder-828805-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 27 03:13:36.392666 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 27 03:13:36.395325 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri May 27 03:13:36 2022] *** May 27 03:13:36.395325 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 27 03:13:36.395325 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 27 03:13:36.395325 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: nodename: dv-cinder-828805-4-iscsi May 27 03:13:36.395325 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: machine: x86_64 May 27 03:13:36.395325 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: clock source: unix May 27 03:13:36.395325 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: pcre jit disabled May 27 03:13:36.395325 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: detected number of CPU cores: 4 May 27 03:13:36.395325 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: current working directory: / May 27 03:13:36.395325 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: detected binary path: /usr/bin/uwsgi-core May 27 03:13:36.395325 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: your processes number limit is 47680 May 27 03:13:36.396080 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: your memory page size is 4096 bytes May 27 03:13:36.396080 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: detected max file descriptor number: 2048 May 27 03:13:36.396080 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: lock engine: pthread robust mutexes May 27 03:13:36.396080 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: thunder lock: enabled May 27 03:13:36.396336 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 27 03:13:36.396336 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 27 03:13:36.440838 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: Python main interpreter initialized at 0x556734bda9c0 May 27 03:13:36.440838 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: python threads support enabled May 27 03:13:36.440838 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: your server socket listen backlog is limited to 100 connections May 27 03:13:36.440838 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: your mercy for graceful operations on workers is 90 seconds May 27 03:13:36.441164 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: mapped 403077 bytes (393 KB) for 2 cores May 27 03:13:36.441246 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: *** Operational MODE: preforking *** May 27 03:13:36.441523 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: *** uWSGI is running in multiple interpreter mode *** May 27 03:13:36.441523 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: spawned uWSGI master process (pid: 80470) May 27 03:13:36.441718 dv-cinder-828805-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 27 03:13:36.442791 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: spawned uWSGI worker 1 (pid: 80481, cores: 1) May 27 03:13:36.442791 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: spawned uWSGI worker 2 (pid: 80482, cores: 1) May 27 03:13:36.442791 dv-cinder-828805-4-iscsi devstack@keystone.service[80470]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 27 03:13:42.140145 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 03:13:42.140145 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: from cryptography.utils import int_from_bytes May 27 03:13:42.140955 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 03:13:42.140955 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: from cryptography.utils import int_from_bytes May 27 03:13:42.141088 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 03:13:42.141088 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: from cryptography.utils import int_from_bytes May 27 03:13:42.141088 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 03:13:42.141088 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: from cryptography.utils import int_from_bytes May 27 03:13:42.189721 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 27 03:13:42.189885 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 27 03:13:42.190076 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] command line args: [] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 27 03:13:42.190166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 27 03:13:42.190276 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 27 03:13:42.190360 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 27 03:13:42.190468 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] command line args: [] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 27 03:13:42.190546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ================================================================================ {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 27 03:13:42.190728 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] admin_token = **** {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.190807 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 27 03:13:42.190924 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ================================================================================ {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 27 03:13:42.191020 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] config_dir = [] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.191166 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.191245 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] admin_token = **** {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.191344 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] config_source = [] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.191428 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] config_dir = [] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.191504 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] debug = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.191584 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.191701 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.192012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] config_source = [] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.192012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] debug = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.192210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.192319 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.192446 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.192558 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.192653 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.192754 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.192861 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.192963 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.193065 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] list_limit = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.193164 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.193265 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] log_config_append = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.193365 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] list_limit = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.193471 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.193573 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] log_config_append = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.193673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] log_dir = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.193773 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.193869 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] log_file = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.193984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] log_dir = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.194071 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.194193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] log_file = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.194268 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.194383 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.194465 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.194536 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.194614 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.194688 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.194808 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.194967 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.195048 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.195166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.195254 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.195327 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.195444 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.195519 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.195640 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.195766 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.195842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.195966 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.196046 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.196163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.196237 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.196363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.196440 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.196556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.196643 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.196715 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.196829 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.196914 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.197164 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.197247 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.24/identity {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.197398 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] publish_errors = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.197628 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.197711 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.24/identity {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.197807 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] publish_errors = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.197888 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.198011 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.198088 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.198253 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.198369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.198483 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.198613 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.198809 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.198960 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] standard_threads = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.199083 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.199197 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.199318 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] standard_threads = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.199433 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.199433 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.199644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.199756 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] use_journal = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.199868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.199993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] use_json = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.200108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.200221 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] use_stderr = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.200340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] use_journal = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.200454 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] use_syslog = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.200574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] use_json = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.200700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.200815 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] use_stderr = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.200815 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] use_syslog = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.201032 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.201153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:13:42.201274 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.201397 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.201515 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.201635 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.201755 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.202123 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.202242 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.202364 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.202478 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.202591 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.202704 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.202816 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.202957 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.203211 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.203338 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.203460 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.203581 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.203697 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.203811 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.203811 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.204042 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.204154 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.204268 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.204382 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.204496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.204688 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.204816 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.204932 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.205047 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.205160 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.205273 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.205400 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.205516 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.205636 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.205757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.205877 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.205992 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.206115 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.206236 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.206356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.206472 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.206586 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.206711 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.206826 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.206975 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.207091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.207205 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.207318 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.207448 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.207565 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.207679 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.207792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.207905 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.208019 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.208140 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.208261 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.208381 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.208497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.208617 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.208737 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.208860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.208975 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.209089 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.209208 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.209322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.209419 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.209520 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.209617 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.209711 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.209805 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.209904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.209999 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.210105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.210238 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.210346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.210463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.210579 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.210702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.210933 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.211045 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.211158 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.211272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.211391 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.211513 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.211643 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.211759 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.211875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.211980 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.212098 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.212219 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.212334 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.212450 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.212573 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.212690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.212805 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.212909 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.213021 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.213144 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.213250 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.213363 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.213484 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.213595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.213709 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.213832 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.213950 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.214064 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.214179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.214301 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.214417 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.214539 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.214656 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.214771 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.214922 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.215038 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.215153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.215280 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.215398 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.215513 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.215634 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.215749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.215871 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.215998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.216115 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.216239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.216361 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.216477 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.216603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.216795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.216923 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.217039 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.217154 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.217269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.217383 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.217506 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.217623 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.217738 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.217853 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.217968 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.218089 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.218213 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.218329 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.218443 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.218558 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.218678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.218794 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.218934 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.219059 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.219175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.219290 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.219404 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.219528 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.219653 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.219769 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.219892 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.220009 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.220125 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.220240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.220354 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.220469 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.220595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.220712 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.220827 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.220942 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.221057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.221181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.221297 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.221418 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.221537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.221654 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.221775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.221899 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.222015 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.222132 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.222247 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.222362 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.222482 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.222606 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.222729 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.222886 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.223015 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.223132 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.223248 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.223371 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.223488 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.223603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.223718 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.223832 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.223947 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.224070 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.224186 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.224303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.224418 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.224533 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.224656 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.224781 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.224898 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.225013 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.225128 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.225251 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.225368 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.225491 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.225609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.225723 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.225838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.225953 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.226067 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] auth.external = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.226190 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.226307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.226421 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.226536 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.226651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] auth.external = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.226766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.226993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.227123 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] auth.password = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.227123 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] auth.token = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.227354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.227471 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.227587 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.227709 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.227827 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] auth.password = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.227951 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.228068 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] auth.token = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.228190 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.228306 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.228429 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.228547 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.228670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.228787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.228911 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.229028 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.229143 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] credential.caching = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.229259 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.229383 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.229502 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.229618 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.229735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.229850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.229965 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] credential.caching = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.230152 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.230271 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.230387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.230504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.230620 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.230736 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.230912 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.231033 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.231150 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.231266 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.231390 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.231522 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.231651 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.231769 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.231885 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.232001 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.232125 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.232243 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.232369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.232487 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.232604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.232719 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.232844 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.232962 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.233087 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] federation.caching = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.233204 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.233320 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.233436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.233552 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.233669 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.233794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.233911 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] federation.caching = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.234033 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.234150 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.234265 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.234902 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.235024 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.235146 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.235263 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.235378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.235494 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.235618 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.235736 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.235853 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.235969 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.236085 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.236201 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.236325 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.236443 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity.caching = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.236559 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.236675 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.236797 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.236921 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.237046 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.237163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.237279 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity.caching = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.237395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.237519 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.237636 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.237761 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.237886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.238002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.238118 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.238223 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.238337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.238462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.238589 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.238705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.238819 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.238986 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.239113 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.239240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.239358 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.239472 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.239588 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.239710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.239832 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.239959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.240086 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.240205 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.240320 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.240436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.240551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.240676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.240793 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.240908 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.241024 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.241140 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.241255 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.241379 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.241500 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.241617 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.241739 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.241862 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.241979 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.242107 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.242224 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.242346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.242481 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.242598 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.242713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.242856 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.242974 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.243091 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.243207 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.243332 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.243449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.243574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.243691 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.243807 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.243922 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.244043 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.244172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.244289 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.244404 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.244520 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.244635 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.244750 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.244872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.244872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.245090 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.245206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.245321 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.245436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.245560 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.245677 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.245792 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.245907 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.246022 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.246137 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.246262 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.246379 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.246494 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.246609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.246725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.246882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.247013 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.247132 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.247248 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.247363 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.247478 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.247592 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.247716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.247833 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.247949 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.248054 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.248167 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.248289 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.248415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.248533 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.248648 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.248763 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.248878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.248992 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.249115 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.249232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.249348 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.249464 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.249585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.249701 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.249825 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.250061 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.250683 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.250800 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.250934 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.251051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.251185 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.251305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.251422 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.251542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.251660 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.251783 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.251909 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.251909 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.252129 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.252245 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.252361 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.252476 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.252476 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.252705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.252821 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.252936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.253053 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.253169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.253293 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.253410 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.253410 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.253628 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.253743 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.253743 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.254037 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.254156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.254272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.254388 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.254503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.254619 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.254825 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.254993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.255110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.255226 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.255342 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.255457 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.255586 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.255706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.255822 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.255937 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.256052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.256167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.256292 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.256409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.256526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.256642 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.256759 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.256875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.257000 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.257118 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.257234 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.257349 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.257465 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.257581 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.257706 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.257831 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.257949 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.258065 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] resource.caching = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.258180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.258180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.258405 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.258524 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.258639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.258754 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.258887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.259005 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] resource.caching = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.259134 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.259252 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.259367 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.259482 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.259597 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.259712 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.259840 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.259840 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.260061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.260176 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.260291 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.260415 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] role.caching = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.260533 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.260649 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] role.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.260764 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.260879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.260996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.261119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.261236 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.261352 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.261474 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] role.caching = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.261595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.261710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] role.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.261834 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.261952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.262069 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.262185 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.262300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.262415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.262540 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.262657 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.262773 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.262927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.263047 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.263163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.263291 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.263410 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.263525 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.263641 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.263757 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.263873 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.263998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.264117 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.264232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.264348 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.264462 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.264578 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.264703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.264821 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.264937 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.265053 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.265170 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.265285 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.265409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.265527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.265665 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.265784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.265901 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.266026 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.266144 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.266261 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.266376 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.266491 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.266606 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.266730 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.266863 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.266980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.267096 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.267213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.267329 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.267457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.267576 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.267692 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.267808 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.267923 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.268038 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.268162 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.268281 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.268397 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.268513 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.268513 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] token.caching = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.268729 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.268853 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.268971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.269086 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.269202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.269318 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.269434 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.269557 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.269674 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.269674 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] token.caching = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.269890 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.269890 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.270107 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.270235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.270353 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.270353 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.270569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.270569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.270784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.270955 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.271074 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.271189 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.271305 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.271428 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.271547 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.271672 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.271789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.271904 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.272020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.272135 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.272250 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.272373 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.272491 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.272491 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.272708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.272823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.272938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.connection = **** {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.273066 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.273195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.273312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.273427 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.273544 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.273659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.273659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.273890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.274008 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.274123 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.connection = **** {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.274311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.274438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.274557 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.274673 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.274788 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.274934 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.275053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.275169 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.275284 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.275399 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.275514 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.275643 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.275761 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.275877 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.275999 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.276115 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.276230 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.276354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.276470 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.276586 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.276702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.276891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.277013 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.277139 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.277256 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.277373 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.277478 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80482) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 27 03:13:42.277592 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.277592 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.277592 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.277592 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.277592 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:13:42.277592 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80481) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 27 03:13:42.303430 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.303651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.303738 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.303871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.303956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.304092 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.312350 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.313149 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.340852 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.341028 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.341332 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.341561 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.393370 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.393706 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.443779 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.444005 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.444710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.444941 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.508782 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.509006 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.509224 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.509348 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.509452 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.509575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.509659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.509793 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.509875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.510009 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.510222 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.510435 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.533253 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.533482 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.533583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.533708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.533795 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.533928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.534011 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.534146 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.534242 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.534354 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.534434 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.534562 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.534644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.534772 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.545668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.546008 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.558638 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.558872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.558975 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.559098 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.559185 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.559311 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.559393 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.559522 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.559616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.559732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.559815 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.559937 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80481) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.560019 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.560141 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80482) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:13:42.579148 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.579352 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 03:13:42.579489 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.579682 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.579765 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 03:13:42.579883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 03:13:42.580035 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.580197 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.580333 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 03:13:42.580466 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.580607 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.580740 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.580870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.580963 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.581038 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.581158 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.592021 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.592158 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.592240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.592427 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.592507 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.592689 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.592770 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.592965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.601433 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.601597 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.604734 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.604885 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.605012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.605157 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.605288 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.605430 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.605560 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.605684 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.605835 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.605962 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.606090 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.606215 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.606462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.606584 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.606735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.606868 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.607031 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.607137 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.607301 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.607403 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.607565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.607664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.634466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.634562 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.636205 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.636290 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.636477 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.636572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.641761 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.641861 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.642030 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.642112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.642303 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.642386 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.649580 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.649700 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.649852 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.649983 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.654316 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.654479 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.654709 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.654894 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.655008 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.655169 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.655283 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.655437 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.655553 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.655712 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.655822 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.655982 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.656086 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.656770 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.656876 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.657034 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.676052 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.676318 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.676400 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.676590 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.680981 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.681303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.684638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.684926 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.685006 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.685250 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.689991 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.690419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.693605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.694058 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.697368 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.697660 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.697850 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.697932 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.698146 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.698227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.698436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.698526 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.698718 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.698808 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.699028 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.699300 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.718036 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.718311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.718570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.719214 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.719372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.719453 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.719684 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.719941 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.727368 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.728027 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.729072 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.729339 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.729739 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.730018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.732402 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.732678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.732949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.733116 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.733219 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.733391 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.733483 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.733665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.733936 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.734206 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.747733 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.748004 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.748276 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.748356 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.748550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.748633 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.748821 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.748901 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.749093 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.749173 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.749366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.749453 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.749641 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.749936 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.767420 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.768121 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.770651 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.771412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.773974 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.774717 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.775595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.776352 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.777211 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.777490 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.777764 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.777983 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.778265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.778539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.785685 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.786499 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.787300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.788144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.790557 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.790872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.791128 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.791393 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.791472 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.791678 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.791955 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.792225 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.801006 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.801282 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.801654 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.801908 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.802191 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.802570 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.809511 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80482) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.809785 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.810058 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.810325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.810496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80481) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:13:42.810577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.810770 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.810879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.811067 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.811149 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.811341 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.811455 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.811617 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.811712 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.811883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.811984 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.812143 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.812233 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.812409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.812528 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.812698 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.812788 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.812971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.813114 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80482) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.813235 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.813476 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.813741 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.814004 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80481) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:13:42.852521 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556734bda9c0 pid: 80482 (default app) May 27 03:13:42.853333 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556734bda9c0 pid: 80481 (default app) May 27 03:13:42.855677 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d871e27-904d-490c-a469-85d610fe6b7d None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:42.855965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d871e27-904d-490c-a469-85d610fe6b7d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:42.856234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d871e27-904d-490c-a469-85d610fe6b7d None None] PATH_INFO: `/v3` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:42.861015 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1/1] 10.222.0.24 () {50 vars in 791 bytes} [Fri May 27 03:13:42 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:51.630492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6551d-961b-49ff-a078-93f7dd7e1017 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:51.631072 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6551d-961b-49ff-a078-93f7dd7e1017 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:51.631072 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd6551d-961b-49ff-a078-93f7dd7e1017 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:51.635484 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1/2] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:13:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:13:51.641083 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543c166-0fbe-4299-8500-990e40d85940 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:51.641469 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543c166-0fbe-4299-8500-990e40d85940 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:51.641624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543c166-0fbe-4299-8500-990e40d85940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:51.708926 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: INFO dbcounter [None req-e543c166-0fbe-4299-8500-990e40d85940 None None] Registered counter for database keystone May 27 03:13:51.729656 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG oslo_db.sqlalchemy.engines [None req-e543c166-0fbe-4299-8500-990e40d85940 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80482) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 27 03:13:51.735842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writer thread running {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 27 03:13:51.908443 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG passlib.handlers.bcrypt [None req-e543c166-0fbe-4299-8500-990e40d85940 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80482) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 27 03:13:51.908775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG passlib.handlers.bcrypt [None req-e543c166-0fbe-4299-8500-990e40d85940 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80482) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 27 03:13:51.964353 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e543c166-0fbe-4299-8500-990e40d85940 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:13:51.971276 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e543c166-0fbe-4299-8500-990e40d85940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:13:52.094206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 2/3] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:13:51 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 27 03:13:52.272127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6551d-961b-49ff-a078-93f7dd7e1017 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:52.340114 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: INFO dbcounter [None req-ecd6551d-961b-49ff-a078-93f7dd7e1017 None None] Registered counter for database keystone May 27 03:13:52.360255 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG oslo_db.sqlalchemy.engines [None req-ecd6551d-961b-49ff-a078-93f7dd7e1017 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80481) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 27 03:13:52.366345 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writer thread running {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 27 03:13:52.484993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd6551d-961b-49ff-a078-93f7dd7e1017 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:52.489335 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173252e3-cf16-4568-93ad-ba82b6c1695e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:52.489995 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173252e3-cf16-4568-93ad-ba82b6c1695e admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:52.490276 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173252e3-cf16-4568-93ad-ba82b6c1695e admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:52.490549 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173252e3-cf16-4568-93ad-ba82b6c1695e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:52.498653 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173252e3-cf16-4568-93ad-ba82b6c1695e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:52.549982 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: WARNING keystone.common.rbac_enforcer.enforcer [None req-173252e3-cf16-4568-93ad-ba82b6c1695e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 27 03:13:52.550312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173252e3-cf16-4568-93ad-ba82b6c1695e admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:52.559760 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 2/4] 10.222.0.24 () {58 vars in 1324 bytes} [Fri May 27 03:13:52 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 03:13:52.565771 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543c166-0fbe-4299-8500-990e40d85940 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:52.580449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543c166-0fbe-4299-8500-990e40d85940 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:52.584917 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf49a65-ad0d-4f80-b942-73594d14f0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:52.586028 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf49a65-ad0d-4f80-b942-73594d14f0cf admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:52.586312 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf49a65-ad0d-4f80-b942-73594d14f0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:52.586588 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf49a65-ad0d-4f80-b942-73594d14f0cf admin admin] PATH_INFO: `/v3/projects/4d3e8df9c2aa4b4588906f73830bd805` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:52.588135 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf49a65-ad0d-4f80-b942-73594d14f0cf admin admin] RBAC: Authorizing `identity:get_project(project_id=4d3e8df9c2aa4b4588906f73830bd805)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:52.746891 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6cf49a65-ad0d-4f80-b942-73594d14f0cf admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 27 03:13:52.747544 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf49a65-ad0d-4f80-b942-73594d14f0cf admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:52.752057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 3/5] 10.222.0.24 () {56 vars in 1207 bytes} [Fri May 27 03:13:52 2022] GET /identity/v3/projects/4d3e8df9c2aa4b4588906f73830bd805 => generated 344 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:54.903608 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac1021-0df6-4866-a355-4912a7a039aa None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:54.904123 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac1021-0df6-4866-a355-4912a7a039aa None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:54.904223 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baac1021-0df6-4866-a355-4912a7a039aa None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:54.908289 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 3/6] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:13:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:13:54.913659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14074514-1a61-4097-a9a8-07315228adf4 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:54.914111 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14074514-1a61-4097-a9a8-07315228adf4 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:54.914244 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14074514-1a61-4097-a9a8-07315228adf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:54.933294 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-14074514-1a61-4097-a9a8-07315228adf4 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:13:54.934894 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-14074514-1a61-4097-a9a8-07315228adf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:13:54.943653 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 4/7] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:13:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 27 03:13:54.965679 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac1021-0df6-4866-a355-4912a7a039aa None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:54.977512 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baac1021-0df6-4866-a355-4912a7a039aa None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:54.981873 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26532d0a-8d16-4219-8862-8600cade4622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:54.982504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26532d0a-8d16-4219-8862-8600cade4622 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:54.982787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26532d0a-8d16-4219-8862-8600cade4622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:54.983097 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26532d0a-8d16-4219-8862-8600cade4622 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:54.991305 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26532d0a-8d16-4219-8862-8600cade4622 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:54.993067 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26532d0a-8d16-4219-8862-8600cade4622 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:55.002263 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 4/8] 10.222.0.24 () {58 vars in 1324 bytes} [Fri May 27 03:13:54 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:13:55.007515 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14074514-1a61-4097-a9a8-07315228adf4 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:55.016296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14074514-1a61-4097-a9a8-07315228adf4 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:55.020578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9026372-3496-4fd1-9d79-0153e10bd02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:55.021431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9026372-3496-4fd1-9d79-0153e10bd02f admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:55.021711 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9026372-3496-4fd1-9d79-0153e10bd02f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:55.021981 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9026372-3496-4fd1-9d79-0153e10bd02f admin admin] PATH_INFO: `/v3/users/93ab1998e69940199adfdba4433a4d08` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:55.023386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9026372-3496-4fd1-9d79-0153e10bd02f admin admin] RBAC: Authorizing `identity:get_user(user_id=93ab1998e69940199adfdba4433a4d08)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:55.025208 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9026372-3496-4fd1-9d79-0153e10bd02f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:55.026742 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 5/9] 10.222.0.24 () {56 vars in 1198 bytes} [Fri May 27 03:13:55 2022] GET /identity/v3/users/93ab1998e69940199adfdba4433a4d08 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:57.271843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6599724-a1e1-4795-9112-5c5556fa6367 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:57.272397 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6599724-a1e1-4795-9112-5c5556fa6367 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:57.272397 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6599724-a1e1-4795-9112-5c5556fa6367 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:57.273731 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 5/10] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:13:57.278880 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9785be-66ac-4a21-a6ee-79d4be9ffe09 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:57.279248 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9785be-66ac-4a21-a6ee-79d4be9ffe09 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:57.279434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9785be-66ac-4a21-a6ee-79d4be9ffe09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:57.292789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425a15c-4604-4595-bac8-9736e0360668 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:57.293073 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425a15c-4604-4595-bac8-9736e0360668 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:57.293351 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7425a15c-4604-4595-bac8-9736e0360668 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:57.294665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 6/11] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:13:57.296332 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-6b9785be-66ac-4a21-a6ee-79d4be9ffe09 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:13:57.297491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6b9785be-66ac-4a21-a6ee-79d4be9ffe09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:13:57.299683 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c68d4b-e7b9-4093-8e05-bf1c7026b63a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:57.299970 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c68d4b-e7b9-4093-8e05-bf1c7026b63a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:57.300251 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c68d4b-e7b9-4093-8e05-bf1c7026b63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:57.339199 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG passlib.handlers.bcrypt [None req-a8c68d4b-e7b9-4093-8e05-bf1c7026b63a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80481) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 27 03:13:57.339553 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG passlib.handlers.bcrypt [None req-a8c68d4b-e7b9-4093-8e05-bf1c7026b63a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80481) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 27 03:13:57.375552 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 6/12] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:13:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:13:57.385847 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-a8c68d4b-e7b9-4093-8e05-bf1c7026b63a None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:13:57.387446 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a8c68d4b-e7b9-4093-8e05-bf1c7026b63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:13:57.398215 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9785be-66ac-4a21-a6ee-79d4be9ffe09 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:57.422135 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:57.487715 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 7/13] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:13:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:13:57.501615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9785be-66ac-4a21-a6ee-79d4be9ffe09 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:57.509423 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c68d4b-e7b9-4093-8e05-bf1c7026b63a None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:57.534756 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff92d8d-4827-4c86-8f00-8e51482a3357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:57.535578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff92d8d-4827-4c86-8f00-8e51482a3357 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:57.535868 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff92d8d-4827-4c86-8f00-8e51482a3357 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:57.536144 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff92d8d-4827-4c86-8f00-8e51482a3357 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:57.547059 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff92d8d-4827-4c86-8f00-8e51482a3357 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:13:57.558548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-8ff92d8d-4827-4c86-8f00-8e51482a3357 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:13:57.558548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:57.558548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:57.558548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:57.558548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:57.558548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:57.558548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:57.558548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:57.558548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:57.558548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:57.558548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:57.558548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:57.558548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:13:57.558548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:13:57.558548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:13:57.558548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:13:57.559838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:13:57.559838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:13:57.559838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:13:57.559838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:13:57.559838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:57.559838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:57.559838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:57.559838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:57.559838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:57.559838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:57.559838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:57.559838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:13:57.559838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:57.559838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:57.559838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:57.559838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:57.560659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:57.560659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:57.560659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:57.560659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:57.560659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:13:57.560659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:13:57.560659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:13:57.560659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:13:57.560659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:13:57.560659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:13:57.560659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:13:57.560659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:13:57.560659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 7/14] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:13:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:13:57.566498 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff92d8d-4827-4c86-8f00-8e51482a3357 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:57.654097 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:57.672490 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff92d8d-4827-4c86-8f00-8e51482a3357 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:57.706965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a954d1c6-e614-4b2c-84e7-381db99a3fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:57.707634 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954d1c6-e614-4b2c-84e7-381db99a3fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:57.707915 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954d1c6-e614-4b2c-84e7-381db99a3fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:57.708185 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954d1c6-e614-4b2c-84e7-381db99a3fd1 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:57.708933 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a954d1c6-e614-4b2c-84e7-381db99a3fd1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:57.709239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a954d1c6-e614-4b2c-84e7-381db99a3fd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:57.710635 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a954d1c6-e614-4b2c-84e7-381db99a3fd1 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:57.717788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c68d4b-e7b9-4093-8e05-bf1c7026b63a None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:57.720136 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:57.722541 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 8/15] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:13:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:13:57.727874 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a954d1c6-e614-4b2c-84e7-381db99a3fd1 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:57.755030 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b25839-67c4-4fa9-bccd-792ea0ee3bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:57.755926 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b25839-67c4-4fa9-bccd-792ea0ee3bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:57.756214 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b25839-67c4-4fa9-bccd-792ea0ee3bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:57.756488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b25839-67c4-4fa9-bccd-792ea0ee3bf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:57.823458 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b25839-67c4-4fa9-bccd-792ea0ee3bf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:57.824983 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b25839-67c4-4fa9-bccd-792ea0ee3bf2 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:57.825528 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a954d1c6-e614-4b2c-84e7-381db99a3fd1 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:57.859511 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da392995-3ce7-4fa0-995e-b54ee3f076f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:57.860365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da392995-3ce7-4fa0-995e-b54ee3f076f4 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:57.860639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da392995-3ce7-4fa0-995e-b54ee3f076f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:57.860905 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da392995-3ce7-4fa0-995e-b54ee3f076f4 None admin] PATH_INFO: `/v3/users/93ab1998e69940199adfdba4433a4d08` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:57.862096 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da392995-3ce7-4fa0-995e-b54ee3f076f4 None admin] RBAC: Authorizing `identity:get_user(user_id=93ab1998e69940199adfdba4433a4d08)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:57.863573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da392995-3ce7-4fa0-995e-b54ee3f076f4 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:57.865180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 9/16] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:13:57 2022] GET /identity/v3/users/93ab1998e69940199adfdba4433a4d08 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:13:57.870372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da392995-3ce7-4fa0-995e-b54ee3f076f4 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:57.884879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:57.923457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 8/17] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:13:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:13:57.928747 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b25839-67c4-4fa9-bccd-792ea0ee3bf2 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:57.931377 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:57.968446 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da392995-3ce7-4fa0-995e-b54ee3f076f4 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:58.002737 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa3110-5839-48be-8595-59b8beebc19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:58.003563 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa3110-5839-48be-8595-59b8beebc19b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:58.003846 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa3110-5839-48be-8595-59b8beebc19b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:58.004114 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa3110-5839-48be-8595-59b8beebc19b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:58.005274 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaa3110-5839-48be-8595-59b8beebc19b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:58.006724 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaa3110-5839-48be-8595-59b8beebc19b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:58.008329 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 10/18] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:13:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:13:58.014023 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa3110-5839-48be-8595-59b8beebc19b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:58.025995 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b25839-67c4-4fa9-bccd-792ea0ee3bf2 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:58.060221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3300f-8495-4c01-bf6e-bc0e35b8ac1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:58.061006 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3300f-8495-4c01-bf6e-bc0e35b8ac1a None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:58.061293 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3300f-8495-4c01-bf6e-bc0e35b8ac1a None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:58.061584 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3300f-8495-4c01-bf6e-bc0e35b8ac1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:58.084335 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f3300f-8495-4c01-bf6e-bc0e35b8ac1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:58.086191 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f3300f-8495-4c01-bf6e-bc0e35b8ac1a None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:58.099442 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:58.107727 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-54f3300f-8495-4c01-bf6e-bc0e35b8ac1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:58.107727 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:58.107727 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:58.107727 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:58.107727 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:58.107727 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:58.107727 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:58.107727 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:58.107727 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:58.107727 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:58.107727 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:58.107727 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:58.107727 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:13:58.107727 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:13:58.107727 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:13:58.122644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:13:58.122644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:58.122644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:58.122644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:58.122644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:58.122644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:58.122644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:58.122644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:58.122644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:13:58.122644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:58.122644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:58.122644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:58.122644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:58.122644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:58.122644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:58.122644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:58.122644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:58.123684 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:13:58.123684 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:13:58.123684 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:58.123684 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:13:58.123684 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 9/19] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:13:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:13:58.123684 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3300f-8495-4c01-bf6e-bc0e35b8ac1a None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:58.124083 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa3110-5839-48be-8595-59b8beebc19b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:58.151545 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d9080-0b1d-4539-89c5-a222e2981a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:58.152202 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d9080-0b1d-4539-89c5-a222e2981a16 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:58.152485 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d9080-0b1d-4539-89c5-a222e2981a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:58.152760 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d9080-0b1d-4539-89c5-a222e2981a16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:58.153590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d9080-0b1d-4539-89c5-a222e2981a16 None admin] RBAC: Adding query filter params (role.id=e5a908bd3ed5445b82a6053246533b88, scope.domain.id=default, user.id=93ab1998e69940199adfdba4433a4d08) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:58.153909 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d9080-0b1d-4539-89c5-a222e2981a16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:58.155402 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d9080-0b1d-4539-89c5-a222e2981a16 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:58.162021 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 11/20] 10.222.0.24 () {56 vars in 1322 bytes} [Fri May 27 03:13:58 2022] GET /identity/v3/role_assignments?user.id=93ab1998e69940199adfdba4433a4d08&scope.domain.id=default&role.id=e5a908bd3ed5445b82a6053246533b88 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:13:58.173041 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:58.220190 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f3300f-8495-4c01-bf6e-bc0e35b8ac1a None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:58.256341 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194a389-44d2-436a-8cfe-b690c7d5d571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:58.257010 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194a389-44d2-436a-8cfe-b690c7d5d571 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:58.257287 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194a389-44d2-436a-8cfe-b690c7d5d571 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:58.257557 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194a389-44d2-436a-8cfe-b690c7d5d571 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:58.258296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194a389-44d2-436a-8cfe-b690c7d5d571 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:58.258597 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194a389-44d2-436a-8cfe-b690c7d5d571 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:58.260065 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194a389-44d2-436a-8cfe-b690c7d5d571 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:58.278433 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 10/21] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:00.938757 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f00e55-121f-4bbd-9e5a-28239435f290 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:00.943234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f00e55-121f-4bbd-9e5a-28239435f290 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:00.943234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f00e55-121f-4bbd-9e5a-28239435f290 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:00.943234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 12/22] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:00.946594 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138167f-2185-4b17-9274-5a103c9b86fd None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:00.947107 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138167f-2185-4b17-9274-5a103c9b86fd None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:00.947181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138167f-2185-4b17-9274-5a103c9b86fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:00.971815 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-5138167f-2185-4b17-9274-5a103c9b86fd None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:00.973337 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5138167f-2185-4b17-9274-5a103c9b86fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:00.986242 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f264453-044d-43a8-aa38-0d583c4588cb None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:00.986826 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f264453-044d-43a8-aa38-0d583c4588cb None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:00.987337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f264453-044d-43a8-aa38-0d583c4588cb None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:00.989156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 13/23] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:00.994784 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc0e14-ff3d-4fb0-8b01-b4619b1b9036 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:00.995342 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc0e14-ff3d-4fb0-8b01-b4619b1b9036 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:00.995825 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc0e14-ff3d-4fb0-8b01-b4619b1b9036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:01.035938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-40cc0e14-ff3d-4fb0-8b01-b4619b1b9036 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:01.037503 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-40cc0e14-ff3d-4fb0-8b01-b4619b1b9036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:01.118860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 11/24] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:01.159925 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138167f-2185-4b17-9274-5a103c9b86fd None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:01.192838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 14/25] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:14:01.193900 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08932339-41bb-4538-9801-cda705a158f9 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:01.194191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08932339-41bb-4538-9801-cda705a158f9 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:01.194465 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08932339-41bb-4538-9801-cda705a158f9 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:01.196069 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 15/26] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:01.201586 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbf0f7-5a5b-46a4-9ff2-480bd51995df None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:01.202172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbf0f7-5a5b-46a4-9ff2-480bd51995df None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:01.202446 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbf0f7-5a5b-46a4-9ff2-480bd51995df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:01.236436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-15cbf0f7-5a5b-46a4-9ff2-480bd51995df None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:01.237670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-15cbf0f7-5a5b-46a4-9ff2-480bd51995df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:01.267216 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:01.321171 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138167f-2185-4b17-9274-5a103c9b86fd None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:01.378220 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 16/27] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:01.379858 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cbf0f7-5a5b-46a4-9ff2-480bd51995df None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:01.383582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:01.468204 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c9450-383f-490b-88f8-22dc23d02b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:01.468940 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c9450-383f-490b-88f8-22dc23d02b53 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:01.469231 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c9450-383f-490b-88f8-22dc23d02b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:01.469511 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c9450-383f-490b-88f8-22dc23d02b53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:01.507149 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cbf0f7-5a5b-46a4-9ff2-480bd51995df None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:01.515856 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105c9450-383f-490b-88f8-22dc23d02b53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:01.524878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-105c9450-383f-490b-88f8-22dc23d02b53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:14:01.524878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:01.524878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:01.524878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:01.524878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:01.524878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:01.524878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:01.524878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:01.524878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:01.524878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:01.524878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:01.524878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:01.524878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:01.524878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:01.524878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:01.524878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:01.526012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:01.526012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:01.526012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:01.526012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:01.526012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:01.526012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:01.526012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:01.526012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:01.526012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:01.526012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:01.526012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:01.526012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:14:01.526012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:01.526012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:01.526012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:01.526012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:01.526970 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:01.526970 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:01.526970 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:01.526970 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:01.526970 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:01.526970 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:01.526970 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:01.526970 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:01.526970 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:01.526970 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:01.526970 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:14:01.526970 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:14:01.530947 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 12/28] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:14:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:01.531805 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c9450-383f-490b-88f8-22dc23d02b53 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:01.547629 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b936089d-73ea-4a4e-bc04-7d4c3875083d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:01.548540 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936089d-73ea-4a4e-bc04-7d4c3875083d None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:01.548998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936089d-73ea-4a4e-bc04-7d4c3875083d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:01.549424 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936089d-73ea-4a4e-bc04-7d4c3875083d None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:01.550313 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b936089d-73ea-4a4e-bc04-7d4c3875083d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:01.551909 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b936089d-73ea-4a4e-bc04-7d4c3875083d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:01.649423 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c9450-383f-490b-88f8-22dc23d02b53 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:01.658443 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:01.687986 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22deec-e60d-45bb-af6e-d7ef6245150d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:01.688797 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22deec-e60d-45bb-af6e-d7ef6245150d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:01.689082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22deec-e60d-45bb-af6e-d7ef6245150d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:01.689361 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d22deec-e60d-45bb-af6e-d7ef6245150d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:01.700538 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22deec-e60d-45bb-af6e-d7ef6245150d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:01.702894 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d22deec-e60d-45bb-af6e-d7ef6245150d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:01.721907 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-1d22deec-e60d-45bb-af6e-d7ef6245150d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:14:01.721907 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:01.721907 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:01.721907 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:01.721907 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:01.721907 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:01.721907 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:01.721907 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:01.721907 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:01.721907 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:01.721907 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:01.721907 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:01.721907 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:14:01.721907 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:14:01.721907 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:14:01.722716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:14:01.722716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:01.722716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:01.722716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:01.722716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:01.722716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:01.722716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:01.722716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:01.722716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:14:01.722716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:01.722716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:01.722716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:01.722716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:01.722716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:01.722716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:01.722716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:01.722716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:01.723527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:14:01.723527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:14:01.723527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:14:01.723527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:14:01.723527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 13/29] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:14:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:01.724337 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22deec-e60d-45bb-af6e-d7ef6245150d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:01.860081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 17/30] 10.222.0.24 () {60 vars in 1131 bytes} [Fri May 27 03:14:01 2022] POST /identity/v3/roles => generated 219 bytes in 481 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:14:01.861385 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b936089d-73ea-4a4e-bc04-7d4c3875083d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:01.903156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d22deec-e60d-45bb-af6e-d7ef6245150d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:01.986866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ffed4-134b-4674-9261-09761b575461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:01.988564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ffed4-134b-4674-9261-09761b575461 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:01.989375 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ffed4-134b-4674-9261-09761b575461 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:01.989957 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ffed4-134b-4674-9261-09761b575461 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:01.991298 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033ffed4-134b-4674-9261-09761b575461 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:01.991961 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033ffed4-134b-4674-9261-09761b575461 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:01.992709 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:01.994851 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033ffed4-134b-4674-9261-09761b575461 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:02.013349 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 14/31] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:02.022753 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae468a-0bf5-47bc-bfad-64ec4fa31df1 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:02.023170 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae468a-0bf5-47bc-bfad-64ec4fa31df1 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:02.023525 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae468a-0bf5-47bc-bfad-64ec4fa31df1 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:02.026488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 15/32] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:02.027849 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a24faa-1306-4cca-b85f-4727d8960d7c None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:02.028215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a24faa-1306-4cca-b85f-4727d8960d7c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:02.028567 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a24faa-1306-4cca-b85f-4727d8960d7c None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:02.034388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 16/33] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:02.035768 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a24faa-1306-4cca-b85f-4727d8960d7c None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:02.048940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b936089d-73ea-4a4e-bc04-7d4c3875083d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:02.064291 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:02.101640 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c2654-e86c-4321-b2fb-2cea7551621a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:02.102311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c2654-e86c-4321-b2fb-2cea7551621a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:02.103649 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c2654-e86c-4321-b2fb-2cea7551621a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:02.103649 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c2654-e86c-4321-b2fb-2cea7551621a None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:02.103847 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c2654-e86c-4321-b2fb-2cea7551621a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:02.104422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c2654-e86c-4321-b2fb-2cea7551621a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:02.105703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c2654-e86c-4321-b2fb-2cea7551621a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:02.126759 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 18/34] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:02.128448 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1474bed-8728-441b-9a33-1f314a04aecb None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:02.128741 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1474bed-8728-441b-9a33-1f314a04aecb None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:02.129022 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1474bed-8728-441b-9a33-1f314a04aecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:02.150171 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d1474bed-8728-441b-9a33-1f314a04aecb None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:02.151846 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d1474bed-8728-441b-9a33-1f314a04aecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:02.165124 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a24faa-1306-4cca-b85f-4727d8960d7c None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:02.205533 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad7d96-c188-40c2-99d1-78d58ec575c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:02.206421 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad7d96-c188-40c2-99d1-78d58ec575c5 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:02.206707 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad7d96-c188-40c2-99d1-78d58ec575c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:02.207017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad7d96-c188-40c2-99d1-78d58ec575c5 None admin] PATH_INFO: `/v3/users/93ab1998e69940199adfdba4433a4d08` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:02.208600 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad7d96-c188-40c2-99d1-78d58ec575c5 None admin] RBAC: Authorizing `identity:get_user(user_id=93ab1998e69940199adfdba4433a4d08)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:02.210040 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad7d96-c188-40c2-99d1-78d58ec575c5 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:02.216135 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 17/35] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:02 2022] GET /identity/v3/users/93ab1998e69940199adfdba4433a4d08 => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:02.216135 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574fb2a-241a-473b-8cb3-1e4f18d1763a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:02.216135 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574fb2a-241a-473b-8cb3-1e4f18d1763a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:02.216135 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574fb2a-241a-473b-8cb3-1e4f18d1763a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:02.234878 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 19/36] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:02.234878 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1474bed-8728-441b-9a33-1f314a04aecb None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:02.239652 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-9574fb2a-241a-473b-8cb3-1e4f18d1763a None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:02.240871 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-9574fb2a-241a-473b-8cb3-1e4f18d1763a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:02.292286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:02.314638 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 18/37] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:02.315688 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9574fb2a-241a-473b-8cb3-1e4f18d1763a None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:02.346176 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:02.350708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1474bed-8728-441b-9a33-1f314a04aecb None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:02.390007 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590c80e-cd2a-4e50-8e7e-ef41a303a54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:02.390679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590c80e-cd2a-4e50-8e7e-ef41a303a54a None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:02.391005 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590c80e-cd2a-4e50-8e7e-ef41a303a54a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:02.391293 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590c80e-cd2a-4e50-8e7e-ef41a303a54a None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:02.392006 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590c80e-cd2a-4e50-8e7e-ef41a303a54a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:02.393491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590c80e-cd2a-4e50-8e7e-ef41a303a54a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:02.440886 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9574fb2a-241a-473b-8cb3-1e4f18d1763a None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:02.475142 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc3cfb-0acf-4f03-9e04-9a6fed7797ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:02.475922 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc3cfb-0acf-4f03-9e04-9a6fed7797ab None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:02.476208 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc3cfb-0acf-4f03-9e04-9a6fed7797ab None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:02.476481 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc3cfb-0acf-4f03-9e04-9a6fed7797ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:02.477650 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc3cfb-0acf-4f03-9e04-9a6fed7797ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:02.479122 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc3cfb-0acf-4f03-9e04-9a6fed7797ab None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:02.480722 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 19/38] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:14:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:02.481554 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc3cfb-0acf-4f03-9e04-9a6fed7797ab None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:02.494685 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 20/39] 10.222.0.24 () {60 vars in 1141 bytes} [Fri May 27 03:14:02 2022] POST /identity/v3/projects => generated 301 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:14:02.495457 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590c80e-cd2a-4e50-8e7e-ef41a303a54a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:02.593096 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:02.609270 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc3cfb-0acf-4f03-9e04-9a6fed7797ab None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:02.619276 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590c80e-cd2a-4e50-8e7e-ef41a303a54a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:02.628217 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:02.649853 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93eda6a-5477-493d-9e1c-ca45573ac601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:02.650541 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93eda6a-5477-493d-9e1c-ca45573ac601 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:02.650825 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93eda6a-5477-493d-9e1c-ca45573ac601 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:02.651130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93eda6a-5477-493d-9e1c-ca45573ac601 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:02.651924 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93eda6a-5477-493d-9e1c-ca45573ac601 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:02.653481 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93eda6a-5477-493d-9e1c-ca45573ac601 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:02.661786 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53479c83-154c-455e-b104-cc4d36377825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:02.662526 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53479c83-154c-455e-b104-cc4d36377825 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:02.662817 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53479c83-154c-455e-b104-cc4d36377825 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:02.663175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53479c83-154c-455e-b104-cc4d36377825 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:02.664116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53479c83-154c-455e-b104-cc4d36377825 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:02.665710 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53479c83-154c-455e-b104-cc4d36377825 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:02.809442 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 21/40] 10.222.0.24 () {60 vars in 1131 bytes} [Fri May 27 03:14:02 2022] POST /identity/v3/roles => generated 223 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:14:02.810260 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53479c83-154c-455e-b104-cc4d36377825 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:02.886118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 20/41] 10.222.0.24 () {60 vars in 1131 bytes} [Fri May 27 03:14:02 2022] POST /identity/v3/roles => generated 225 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:14:02.888035 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652c4ea-326e-4de6-8e29-9a37963096d9 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:02.888325 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652c4ea-326e-4de6-8e29-9a37963096d9 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:02.888661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652c4ea-326e-4de6-8e29-9a37963096d9 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:02.890810 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 21/42] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:02.902953 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada405ca-d6c3-4d6c-ad30-cb409f7e88c4 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:02.902953 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada405ca-d6c3-4d6c-ad30-cb409f7e88c4 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:02.902953 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada405ca-d6c3-4d6c-ad30-cb409f7e88c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:02.927127 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53479c83-154c-455e-b104-cc4d36377825 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:02.930347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-ada405ca-d6c3-4d6c-ad30-cb409f7e88c4 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:02.932122 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ada405ca-d6c3-4d6c-ad30-cb409f7e88c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:02.971037 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78989c14-f62f-4164-9991-b1c4ea1a0bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:02.971955 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78989c14-f62f-4164-9991-b1c4ea1a0bc0 None admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:02.972251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78989c14-f62f-4164-9991-b1c4ea1a0bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:02.972547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78989c14-f62f-4164-9991-b1c4ea1a0bc0 None admin] PATH_INFO: `/v3/domains/default/users/93ab1998e69940199adfdba4433a4d08/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:02.974766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78989c14-f62f-4164-9991-b1c4ea1a0bc0 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=93ab1998e69940199adfdba4433a4d08, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:02.976508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78989c14-f62f-4164-9991-b1c4ea1a0bc0 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:03.018647 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 22/43] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:03.047027 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada405ca-d6c3-4d6c-ad30-cb409f7e88c4 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:03.052677 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 22/44] 10.222.0.24 () {58 vars in 1361 bytes} [Fri May 27 03:14:02 2022] PUT /identity/v3/domains/default/users/93ab1998e69940199adfdba4433a4d08/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:14:03.150065 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:03.175765 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada405ca-d6c3-4d6c-ad30-cb409f7e88c4 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:03.213141 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f30d1-2546-4eda-8497-54cc7d3858b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:03.213949 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f30d1-2546-4eda-8497-54cc7d3858b2 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:03.214231 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f30d1-2546-4eda-8497-54cc7d3858b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:03.214503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f30d1-2546-4eda-8497-54cc7d3858b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:03.215779 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f30d1-2546-4eda-8497-54cc7d3858b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:03.217410 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f30d1-2546-4eda-8497-54cc7d3858b2 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:03.219699 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 23/45] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:14:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:03.225413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78989c14-f62f-4164-9991-b1c4ea1a0bc0 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:03.256655 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:03.328594 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78989c14-f62f-4164-9991-b1c4ea1a0bc0 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:03.361107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2d7fb-be3f-4f1b-9b83-b3601e2bbe83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:03.361765 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2d7fb-be3f-4f1b-9b83-b3601e2bbe83 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:03.362047 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2d7fb-be3f-4f1b-9b83-b3601e2bbe83 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:03.362323 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2d7fb-be3f-4f1b-9b83-b3601e2bbe83 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:03.363077 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2d7fb-be3f-4f1b-9b83-b3601e2bbe83 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:03.364524 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2d7fb-be3f-4f1b-9b83-b3601e2bbe83 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:03.446518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 23/46] 10.222.0.24 () {60 vars in 1141 bytes} [Fri May 27 03:14:03 2022] POST /identity/v3/projects => generated 312 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:14:05.259350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c373af-168f-41ba-85a2-9acc3e975eca None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:05.259880 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c373af-168f-41ba-85a2-9acc3e975eca None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:05.259880 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c373af-168f-41ba-85a2-9acc3e975eca None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:05.261256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 24/47] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:05.267080 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e39989-2390-4235-85ae-4897a57eb4f4 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:05.267457 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e39989-2390-4235-85ae-4897a57eb4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:05.267636 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e39989-2390-4235-85ae-4897a57eb4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:05.283991 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-82e39989-2390-4235-85ae-4897a57eb4f4 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:05.285182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-82e39989-2390-4235-85ae-4897a57eb4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:05.348950 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 24/48] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:05.371807 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c373af-168f-41ba-85a2-9acc3e975eca None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:05.495037 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c373af-168f-41ba-85a2-9acc3e975eca None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:05.530501 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ede9d-a316-401c-a79d-af57cbd269c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:05.531328 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ede9d-a316-401c-a79d-af57cbd269c5 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:05.531618 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ede9d-a316-401c-a79d-af57cbd269c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:05.531898 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ede9d-a316-401c-a79d-af57cbd269c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:05.541988 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39ede9d-a316-401c-a79d-af57cbd269c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:05.550168 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-e39ede9d-a316-401c-a79d-af57cbd269c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:14:05.550168 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:05.550168 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:05.550168 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:05.550168 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:05.550168 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:05.550168 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:05.550168 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:05.550168 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:05.550168 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:05.550168 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:05.550168 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:05.550168 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:05.550168 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:05.550168 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:05.550168 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:05.551112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:05.551112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:05.551112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:05.551112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:05.551112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:05.551112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:05.551112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:05.551112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:05.551112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:05.551112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:05.551112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:05.551112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:14:05.551112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:05.551112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:05.551112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:05.551112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:05.551850 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:05.551850 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:05.551850 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:05.551850 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:05.551850 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:05.551850 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:05.551850 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:05.551850 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:05.551850 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:05.551850 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:05.551850 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:14:05.551850 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:14:05.551850 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 25/49] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:14:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:05.557262 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e39989-2390-4235-85ae-4897a57eb4f4 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:05.567812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:05.641230 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c88af-ab67-493a-8a12-1eed7cbba5f1 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:05.641440 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c88af-ab67-493a-8a12-1eed7cbba5f1 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:05.641725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c88af-ab67-493a-8a12-1eed7cbba5f1 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:05.643132 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 26/50] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:14:05.648033 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5e242-b87b-4bd8-ba28-7d2f6450a903 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:05.648325 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5e242-b87b-4bd8-ba28-7d2f6450a903 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:05.648606 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5e242-b87b-4bd8-ba28-7d2f6450a903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:05.653009 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e39989-2390-4235-85ae-4897a57eb4f4 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:05.654576 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:05.665409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-96e5e242-b87b-4bd8-ba28-7d2f6450a903 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:05.666597 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-96e5e242-b87b-4bd8-ba28-7d2f6450a903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:05.688610 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b2f0b-2c48-4b33-9150-ad0b22856e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:05.689288 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b2f0b-2c48-4b33-9150-ad0b22856e92 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:05.689570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b2f0b-2c48-4b33-9150-ad0b22856e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:05.689843 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984b2f0b-2c48-4b33-9150-ad0b22856e92 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:05.690592 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b2f0b-2c48-4b33-9150-ad0b22856e92 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:05.690924 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b2f0b-2c48-4b33-9150-ad0b22856e92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:05.692317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984b2f0b-2c48-4b33-9150-ad0b22856e92 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:05.700240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 25/51] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:05.706672 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b2f0b-2c48-4b33-9150-ad0b22856e92 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:05.734673 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 27/52] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:14:05.756929 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5e242-b87b-4bd8-ba28-7d2f6450a903 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:05.773145 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:05.804873 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984b2f0b-2c48-4b33-9150-ad0b22856e92 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:05.837331 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5f923-6f9d-4d65-b445-6f9c9bf1cd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:05.838235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5f923-6f9d-4d65-b445-6f9c9bf1cd37 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:05.838517 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5f923-6f9d-4d65-b445-6f9c9bf1cd37 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:05.838793 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5f923-6f9d-4d65-b445-6f9c9bf1cd37 None admin] PATH_INFO: `/v3/users/93ab1998e69940199adfdba4433a4d08` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:05.840104 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add5f923-6f9d-4d65-b445-6f9c9bf1cd37 None admin] RBAC: Authorizing `identity:get_user(user_id=93ab1998e69940199adfdba4433a4d08)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:05.841595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add5f923-6f9d-4d65-b445-6f9c9bf1cd37 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:05.844115 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 26/53] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:05 2022] GET /identity/v3/users/93ab1998e69940199adfdba4433a4d08 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:05.849362 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5f923-6f9d-4d65-b445-6f9c9bf1cd37 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:05.856392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5e242-b87b-4bd8-ba28-7d2f6450a903 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:05.891385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4b81a-f9f9-4c37-8792-f6161bc09a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:05.892100 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4b81a-f9f9-4c37-8792-f6161bc09a89 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:05.892382 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4b81a-f9f9-4c37-8792-f6161bc09a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:05.892659 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4b81a-f9f9-4c37-8792-f6161bc09a89 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:05.893822 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b4b81a-f9f9-4c37-8792-f6161bc09a89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:05.895300 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b4b81a-f9f9-4c37-8792-f6161bc09a89 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:05.897062 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 28/54] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:14:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:05.902314 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4b81a-f9f9-4c37-8792-f6161bc09a89 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:05.905762 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:05.948759 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5f923-6f9d-4d65-b445-6f9c9bf1cd37 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:05.982337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d8554-dd44-4eba-a176-3b4ab8a341b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:05.983223 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d8554-dd44-4eba-a176-3b4ab8a341b1 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:05.984017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d8554-dd44-4eba-a176-3b4ab8a341b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:05.984296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49d8554-dd44-4eba-a176-3b4ab8a341b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:05.985470 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d8554-dd44-4eba-a176-3b4ab8a341b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:05.986935 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49d8554-dd44-4eba-a176-3b4ab8a341b1 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:05.988539 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 27/55] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:14:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:05.994186 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d8554-dd44-4eba-a176-3b4ab8a341b1 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:06.002702 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4b81a-f9f9-4c37-8792-f6161bc09a89 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:06.003039 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:06.036151 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17e80d-2014-4ef2-a9aa-09cef6faafa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:06.036816 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17e80d-2014-4ef2-a9aa-09cef6faafa1 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:06.037103 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17e80d-2014-4ef2-a9aa-09cef6faafa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:06.037388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17e80d-2014-4ef2-a9aa-09cef6faafa1 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:06.038110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17e80d-2014-4ef2-a9aa-09cef6faafa1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:06.039625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17e80d-2014-4ef2-a9aa-09cef6faafa1 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:06.092186 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49d8554-dd44-4eba-a176-3b4ab8a341b1 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:06.125778 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f64ff-4a55-40f5-9c3c-0da1ecd18092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:06.126436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f64ff-4a55-40f5-9c3c-0da1ecd18092 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:06.126724 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f64ff-4a55-40f5-9c3c-0da1ecd18092 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:06.127037 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f64ff-4a55-40f5-9c3c-0da1ecd18092 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:06.127870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f64ff-4a55-40f5-9c3c-0da1ecd18092 None admin] RBAC: Adding query filter params (role.id=e5a908bd3ed5445b82a6053246533b88, scope.domain.id=default, user.id=93ab1998e69940199adfdba4433a4d08) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:06.128184 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f64ff-4a55-40f5-9c3c-0da1ecd18092 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:06.129632 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f64ff-4a55-40f5-9c3c-0da1ecd18092 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:06.135341 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 28/56] 10.222.0.24 () {56 vars in 1322 bytes} [Fri May 27 03:14:05 2022] GET /identity/v3/role_assignments?user.id=93ab1998e69940199adfdba4433a4d08&scope.domain.id=default&role.id=e5a908bd3ed5445b82a6053246533b88 => generated 530 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:06.148417 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 29/57] 10.222.0.24 () {60 vars in 1140 bytes} [Fri May 27 03:14:05 2022] POST /identity/v3/projects => generated 298 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:14:08.328747 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238e14f-9431-4a31-919c-0e11f882a715 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:08.329230 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238e14f-9431-4a31-919c-0e11f882a715 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:08.329230 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2238e14f-9431-4a31-919c-0e11f882a715 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:08.330595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 29/58] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:08.335628 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc89f0-d67e-4ad4-a017-f869fcb94b4f None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:08.335991 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc89f0-d67e-4ad4-a017-f869fcb94b4f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:08.336186 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc89f0-d67e-4ad4-a017-f869fcb94b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:08.352823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-dfbc89f0-d67e-4ad4-a017-f869fcb94b4f None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:08.353980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-dfbc89f0-d67e-4ad4-a017-f869fcb94b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:08.402546 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:08.419859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 30/59] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:08.443018 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2238e14f-9431-4a31-919c-0e11f882a715 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:08.511271 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:08.542531 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2238e14f-9431-4a31-919c-0e11f882a715 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:08.574742 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f581d-c60d-43ad-a4ac-ce85495fa5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:08.575548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f581d-c60d-43ad-a4ac-ce85495fa5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:08.575832 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f581d-c60d-43ad-a4ac-ce85495fa5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:08.576108 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f581d-c60d-43ad-a4ac-ce85495fa5f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:08.578002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f581d-c60d-43ad-a4ac-ce85495fa5f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:08.579713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f581d-c60d-43ad-a4ac-ce85495fa5f8 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:08.581246 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 30/60] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:14:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:08.586592 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc89f0-d67e-4ad4-a017-f869fcb94b4f None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:08.681581 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc89f0-d67e-4ad4-a017-f869fcb94b4f None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:08.713838 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fee25-8971-4a22-b1cb-9e941b42f785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:08.714484 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fee25-8971-4a22-b1cb-9e941b42f785 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:08.714765 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fee25-8971-4a22-b1cb-9e941b42f785 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:08.715061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fee25-8971-4a22-b1cb-9e941b42f785 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:08.715765 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4fee25-8971-4a22-b1cb-9e941b42f785 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:08.717230 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4fee25-8971-4a22-b1cb-9e941b42f785 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:08.805512 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 31/61] 10.222.0.24 () {60 vars in 1132 bytes} [Fri May 27 03:14:08 2022] POST /identity/v3/users => generated 275 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:14:12.125014 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cc6aa-ed8b-4885-95ac-f5ab72380400 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:12.125519 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cc6aa-ed8b-4885-95ac-f5ab72380400 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:12.125592 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cc6aa-ed8b-4885-95ac-f5ab72380400 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:12.126943 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 31/62] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:12.132617 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a5bf4-5d3c-40a9-a0cb-ddd7f83186e4 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:12.133052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a5bf4-5d3c-40a9-a0cb-ddd7f83186e4 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:12.133161 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a5bf4-5d3c-40a9-a0cb-ddd7f83186e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:12.160007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-6c3a5bf4-5d3c-40a9-a0cb-ddd7f83186e4 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:12.161678 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6c3a5bf4-5d3c-40a9-a0cb-ddd7f83186e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:12.220787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:12.268292 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 32/63] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:12.303442 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8cc6aa-ed8b-4885-95ac-f5ab72380400 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:12.324862 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74140291-7e23-4f35-a721-c3666a4f74f8 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:12.325186 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74140291-7e23-4f35-a721-c3666a4f74f8 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:12.325428 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74140291-7e23-4f35-a721-c3666a4f74f8 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:12.327480 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 33/64] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:12.343762 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eaa1c0-ebc4-4e37-9914-17e81b3021b9 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:12.344067 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eaa1c0-ebc4-4e37-9914-17e81b3021b9 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:12.344335 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eaa1c0-ebc4-4e37-9914-17e81b3021b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:12.376087 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-f5eaa1c0-ebc4-4e37-9914-17e81b3021b9 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:12.383169 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f5eaa1c0-ebc4-4e37-9914-17e81b3021b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:12.454140 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 34/65] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 27 03:14:12.456102 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079fd21-9427-4ae0-a391-4a34fa8fbe6a None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:12.456548 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079fd21-9427-4ae0-a391-4a34fa8fbe6a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:12.456973 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079fd21-9427-4ae0-a391-4a34fa8fbe6a None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:12.458564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 35/66] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:12.464547 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd807a-91a5-4851-8b32-26f885f0b9ac None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:12.465057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd807a-91a5-4851-8b32-26f885f0b9ac None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:12.465480 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd807a-91a5-4851-8b32-26f885f0b9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:12.468921 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8cc6aa-ed8b-4885-95ac-f5ab72380400 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:12.485285 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-56bd807a-91a5-4851-8b32-26f885f0b9ac None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:12.486743 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-56bd807a-91a5-4851-8b32-26f885f0b9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:12.508487 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24121c9-2822-41f6-afec-65b309ee0ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:12.509302 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24121c9-2822-41f6-afec-65b309ee0ead None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:12.509585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24121c9-2822-41f6-afec-65b309ee0ead None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:12.509862 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24121c9-2822-41f6-afec-65b309ee0ead None admin] PATH_INFO: `/v3/roles/member` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:12.518211 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:12.523757 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24121c9-2822-41f6-afec-65b309ee0ead None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:12.531719 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-f24121c9-2822-41f6-afec-65b309ee0ead None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:12.531719 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:12.531719 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:12.531719 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:12.531719 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:12.531719 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:12.531719 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:12.531719 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:12.531719 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:12.531719 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:12.531719 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:12.531719 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:12.531719 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:12.531719 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:12.531719 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:12.531719 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:12.533269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:12.533269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:12.533269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:12.533269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:12.533269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:12.533269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:12.533269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:12.533269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:12.533269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:12.533269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:12.533269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:12.533269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:14:12.533269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:12.533269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:12.533269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:12.533269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:12.534051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:12.534051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:12.534051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:12.534051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:12.534051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:12.534051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:12.534051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:12.534051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:12.534051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:12.534051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:12.534051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:12.534051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:14:12.534051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 32/67] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:12.534051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24121c9-2822-41f6-afec-65b309ee0ead None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:12.565887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 36/68] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:14:12.567009 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd807a-91a5-4851-8b32-26f885f0b9ac None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:12.600172 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:12.670638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24121c9-2822-41f6-afec-65b309ee0ead None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:12.688203 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd807a-91a5-4851-8b32-26f885f0b9ac None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:12.707760 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f75ff0-abdf-422b-b4bd-b20b147cadbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:12.708764 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f75ff0-abdf-422b-b4bd-b20b147cadbb None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:12.709055 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f75ff0-abdf-422b-b4bd-b20b147cadbb None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:12.709331 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f75ff0-abdf-422b-b4bd-b20b147cadbb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:12.720585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f75ff0-abdf-422b-b4bd-b20b147cadbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:12.726634 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26a4b95-bdbb-4123-b85d-f360cf454c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:12.727336 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a4b95-bdbb-4123-b85d-f360cf454c67 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:12.727618 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a4b95-bdbb-4123-b85d-f360cf454c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:12.727893 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26a4b95-bdbb-4123-b85d-f360cf454c67 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:12.728644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26a4b95-bdbb-4123-b85d-f360cf454c67 None admin] RBAC: Adding query filter params (name=member) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:12.728735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-96f75ff0-abdf-422b-b4bd-b20b147cadbb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:14:12.728735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:12.728735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:12.728735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:12.728735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:12.728735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:12.728735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:12.728735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:12.728735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:12.728735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:12.728735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:12.728735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:12.728735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:12.728735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:12.728735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:12.728735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:12.729512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:12.729512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:12.729512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:12.729512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:12.729512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:12.729512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:12.729512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:12.729512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:12.729512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:12.729512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:12.729512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:12.729512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:14:12.729512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:12.729512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:12.729512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:12.729512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:12.730260 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:12.730260 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:12.730260 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:12.730260 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:12.730260 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:12.730260 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:12.730260 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:12.730260 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:12.730260 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:12.730260 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:12.730260 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:14:12.730260 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:14:12.730824 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26a4b95-bdbb-4123-b85d-f360cf454c67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:12.730824 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26a4b95-bdbb-4123-b85d-f360cf454c67 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:12.734685 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 33/69] 10.222.0.24 () {56 vars in 1114 bytes} [Fri May 27 03:14:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:12.735565 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f75ff0-abdf-422b-b4bd-b20b147cadbb None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:12.739629 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 37/70] 10.222.0.24 () {56 vars in 1101 bytes} [Fri May 27 03:14:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:12.740948 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4adfd-2cf9-4184-accf-d316d89e8e29 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:12.741240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4adfd-2cf9-4184-accf-d316d89e8e29 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:12.741508 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4adfd-2cf9-4184-accf-d316d89e8e29 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:12.742949 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 38/71] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:12.747500 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4adfd-2cf9-4184-accf-d316d89e8e29 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:12.802969 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:12.845792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f75ff0-abdf-422b-b4bd-b20b147cadbb None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:12.849540 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:12.860716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4adfd-2cf9-4184-accf-d316d89e8e29 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:12.880663 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d10fc1-fc8e-45a2-928d-71ecad36501d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:12.881461 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d10fc1-fc8e-45a2-928d-71ecad36501d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:12.881864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d10fc1-fc8e-45a2-928d-71ecad36501d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:12.882234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d10fc1-fc8e-45a2-928d-71ecad36501d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:12.893451 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d10fc1-fc8e-45a2-928d-71ecad36501d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:12.896998 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4ce65-1c39-4a92-8bb7-a00fabf12e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:12.897669 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4ce65-1c39-4a92-8bb7-a00fabf12e94 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:12.897955 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4ce65-1c39-4a92-8bb7-a00fabf12e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:12.898232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4ce65-1c39-4a92-8bb7-a00fabf12e94 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:12.899020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4ce65-1c39-4a92-8bb7-a00fabf12e94 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:12.899331 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4ce65-1c39-4a92-8bb7-a00fabf12e94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:12.900703 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4ce65-1c39-4a92-8bb7-a00fabf12e94 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:12.901387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-20d10fc1-fc8e-45a2-928d-71ecad36501d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:14:12.901387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:12.901387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:12.901387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:12.901387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:12.901387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:12.901387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:12.901387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:12.901387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:12.901387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:12.901387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:12.901387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:12.901387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:12.901387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:12.901387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:12.901387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:12.902311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:12.902311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:12.902311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:12.902311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:12.902311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:12.902311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:12.902311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:12.902311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:12.902311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:12.902311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:12.902311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:12.902311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:14:12.902311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:12.902311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:12.902311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:12.902311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:12.903319 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:12.903319 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:12.903319 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:12.903319 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:12.903319 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:12.903319 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:12.903319 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:12.903319 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:12.903319 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:12.903319 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:12.903319 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:14:12.903319 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:14:12.907441 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 34/72] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:14:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:12.908330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d10fc1-fc8e-45a2-928d-71ecad36501d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:12.909480 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 39/73] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:14:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:12.910879 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0430eeb-9714-4f0c-a7da-2a273c6640d3 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:12.911170 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0430eeb-9714-4f0c-a7da-2a273c6640d3 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:12.911446 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0430eeb-9714-4f0c-a7da-2a273c6640d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:12.928340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-f0430eeb-9714-4f0c-a7da-2a273c6640d3 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:12.929593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f0430eeb-9714-4f0c-a7da-2a273c6640d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:12.996229 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 40/74] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:12.997714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66883a4a-f65b-4f38-824d-f1842a07baae None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:12.998010 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66883a4a-f65b-4f38-824d-f1842a07baae None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:12.998281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66883a4a-f65b-4f38-824d-f1842a07baae None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:12.999606 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 41/75] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:13.004661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66883a4a-f65b-4f38-824d-f1842a07baae None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:13.010563 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d10fc1-fc8e-45a2-928d-71ecad36501d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:13.032379 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:13.045981 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0d26a-3465-43fc-9fa3-1abdaec5eec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:13.046889 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0d26a-3465-43fc-9fa3-1abdaec5eec7 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:13.047175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0d26a-3465-43fc-9fa3-1abdaec5eec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:13.047455 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0d26a-3465-43fc-9fa3-1abdaec5eec7 None admin] PATH_INFO: `/v3/users/3e25056b9cac4b5cb4ff28c737a1cbdd` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:13.060978 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0d26a-3465-43fc-9fa3-1abdaec5eec7 None admin] RBAC: Authorizing `identity:get_user(user_id=3e25056b9cac4b5cb4ff28c737a1cbdd)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:13.062472 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0d26a-3465-43fc-9fa3-1abdaec5eec7 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:13.066981 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 35/76] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:12 2022] GET /identity/v3/users/3e25056b9cac4b5cb4ff28c737a1cbdd => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:13.067830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0d26a-3465-43fc-9fa3-1abdaec5eec7 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:13.098281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:13.115411 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66883a4a-f65b-4f38-824d-f1842a07baae None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:13.150363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d267cf0-f2a0-4ecf-9fc0-5e4747ca28df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:13.151161 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d267cf0-f2a0-4ecf-9fc0-5e4747ca28df None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:13.151348 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d267cf0-f2a0-4ecf-9fc0-5e4747ca28df None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:13.151624 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d267cf0-f2a0-4ecf-9fc0-5e4747ca28df None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:13.152377 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d267cf0-f2a0-4ecf-9fc0-5e4747ca28df None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:13.152684 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d267cf0-f2a0-4ecf-9fc0-5e4747ca28df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:13.154087 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d267cf0-f2a0-4ecf-9fc0-5e4747ca28df None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:13.163063 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 42/77] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:13.164282 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313eedac-82c0-4a5b-8402-ca339949c8c7 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:13.164570 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313eedac-82c0-4a5b-8402-ca339949c8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:13.164847 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313eedac-82c0-4a5b-8402-ca339949c8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:13.174757 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0d26a-3465-43fc-9fa3-1abdaec5eec7 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:13.181166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-313eedac-82c0-4a5b-8402-ca339949c8c7 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:13.182382 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-313eedac-82c0-4a5b-8402-ca339949c8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:13.209348 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90725132-0c17-44a4-8149-628e6acef41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:13.210221 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90725132-0c17-44a4-8149-628e6acef41c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:13.210505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90725132-0c17-44a4-8149-628e6acef41c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:13.210777 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90725132-0c17-44a4-8149-628e6acef41c None admin] PATH_INFO: `/v3/users/3e25056b9cac4b5cb4ff28c737a1cbdd` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:13.212065 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90725132-0c17-44a4-8149-628e6acef41c None admin] RBAC: Authorizing `identity:get_user(user_id=3e25056b9cac4b5cb4ff28c737a1cbdd)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:13.213710 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90725132-0c17-44a4-8149-628e6acef41c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:13.215377 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 36/78] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:13 2022] GET /identity/v3/users/3e25056b9cac4b5cb4ff28c737a1cbdd => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:13.216149 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90725132-0c17-44a4-8149-628e6acef41c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:13.255071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 43/79] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:13.255940 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313eedac-82c0-4a5b-8402-ca339949c8c7 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:13.272676 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:13.329882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90725132-0c17-44a4-8149-628e6acef41c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:13.336252 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:13.361551 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313eedac-82c0-4a5b-8402-ca339949c8c7 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:13.365504 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083a072-c138-4701-bf8f-adae405a1ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:13.366244 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083a072-c138-4701-bf8f-adae405a1ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:13.366526 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083a072-c138-4701-bf8f-adae405a1ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:13.366918 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083a072-c138-4701-bf8f-adae405a1ba2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:13.378899 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083a072-c138-4701-bf8f-adae405a1ba2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:13.386624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-e083a072-c138-4701-bf8f-adae405a1ba2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:13.386624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:13.386624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:13.386624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:13.386624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:13.386624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:13.386624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:13.386624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:13.386624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:13.386624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:13.386624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:13.386624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:13.386624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:13.386624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:13.386624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:13.386624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:13.387505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:13.387505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:13.387505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:13.387505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:13.387505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:13.387505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:13.387505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:13.387505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:13.387505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:13.387505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:13.387505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:13.387505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:14:13.387505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:13.387505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:13.387505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:13.387505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:13.388243 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:13.388243 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:13.388243 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:13.388243 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:13.388243 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:13.388243 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:13.388243 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:13.388243 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:13.388243 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:13.388243 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:13.388243 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:13.388243 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:14:13.388243 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 37/80] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:13.393452 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083a072-c138-4701-bf8f-adae405a1ba2 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:13.396681 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bede4-2ce1-495c-877b-40d5fc8210df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:13.397332 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bede4-2ce1-495c-877b-40d5fc8210df None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:13.397616 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bede4-2ce1-495c-877b-40d5fc8210df None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:13.397889 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bede4-2ce1-495c-877b-40d5fc8210df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:13.465109 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66bede4-2ce1-495c-877b-40d5fc8210df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:13.466563 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66bede4-2ce1-495c-877b-40d5fc8210df None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:13.492569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083a072-c138-4701-bf8f-adae405a1ba2 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:13.494891 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:13.527172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e53a4-2fcf-4ddb-b76e-a6917f4033c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:13.528046 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e53a4-2fcf-4ddb-b76e-a6917f4033c6 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:13.528329 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e53a4-2fcf-4ddb-b76e-a6917f4033c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:13.528605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e53a4-2fcf-4ddb-b76e-a6917f4033c6 None admin] PATH_INFO: `/v3/users/93ab1998e69940199adfdba4433a4d08` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:13.530350 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e53a4-2fcf-4ddb-b76e-a6917f4033c6 None admin] RBAC: Authorizing `identity:get_user(user_id=93ab1998e69940199adfdba4433a4d08)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:13.531836 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e53a4-2fcf-4ddb-b76e-a6917f4033c6 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:13.533652 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 38/81] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:13 2022] GET /identity/v3/users/93ab1998e69940199adfdba4433a4d08 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:13.534224 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e53a4-2fcf-4ddb-b76e-a6917f4033c6 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:13.540106 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:13.566300 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 44/82] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:14:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:14:13.567186 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bede4-2ce1-495c-877b-40d5fc8210df None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:13.642968 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e53a4-2fcf-4ddb-b76e-a6917f4033c6 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:13.665797 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bede4-2ce1-495c-877b-40d5fc8210df None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:13.675475 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85f458-b765-4127-a94d-ebb7a7ad0c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:13.676122 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85f458-b765-4127-a94d-ebb7a7ad0c57 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:13.676402 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85f458-b765-4127-a94d-ebb7a7ad0c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:13.676677 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85f458-b765-4127-a94d-ebb7a7ad0c57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:13.699805 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70dfd9-510c-4282-9b18-e036c9ca5780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:13.700601 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70dfd9-510c-4282-9b18-e036c9ca5780 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:13.700889 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70dfd9-510c-4282-9b18-e036c9ca5780 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:13.701181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70dfd9-510c-4282-9b18-e036c9ca5780 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:13.703197 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e70dfd9-510c-4282-9b18-e036c9ca5780 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:13.705058 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e70dfd9-510c-4282-9b18-e036c9ca5780 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:13.706737 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 45/83] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:14:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:13.707532 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70dfd9-510c-4282-9b18-e036c9ca5780 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:13.716511 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:13.742584 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85f458-b765-4127-a94d-ebb7a7ad0c57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:13.744097 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85f458-b765-4127-a94d-ebb7a7ad0c57 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:13.753710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:13.807602 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70dfd9-510c-4282-9b18-e036c9ca5780 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:13.841844 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeccb9d-8c0e-4311-81fe-37d598d69edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:13.843262 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeccb9d-8c0e-4311-81fe-37d598d69edd None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:13.843551 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeccb9d-8c0e-4311-81fe-37d598d69edd None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:13.843830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeccb9d-8c0e-4311-81fe-37d598d69edd None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:13.844494 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 39/84] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:14:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:14:13.844635 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeccb9d-8c0e-4311-81fe-37d598d69edd None admin] RBAC: Adding query filter params (name=member) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:13.844884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeccb9d-8c0e-4311-81fe-37d598d69edd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:13.845338 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85f458-b765-4127-a94d-ebb7a7ad0c57 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:13.846286 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeccb9d-8c0e-4311-81fe-37d598d69edd None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:13.858816 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 46/85] 10.222.0.24 () {56 vars in 1101 bytes} [Fri May 27 03:14:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:13.858816 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeccb9d-8c0e-4311-81fe-37d598d69edd None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:13.913909 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:13.942296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85f458-b765-4127-a94d-ebb7a7ad0c57 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:13.954856 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeccb9d-8c0e-4311-81fe-37d598d69edd None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:13.961692 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:13.976881 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6cdb1-cc20-4af7-8fee-192bb30e206d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:13.977535 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6cdb1-cc20-4af7-8fee-192bb30e206d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:13.977823 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6cdb1-cc20-4af7-8fee-192bb30e206d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:13.978114 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6cdb1-cc20-4af7-8fee-192bb30e206d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:13.987364 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eda3c4-9cd1-4bfb-a895-460f2fbb3380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:13.988511 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eda3c4-9cd1-4bfb-a895-460f2fbb3380 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:13.988511 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eda3c4-9cd1-4bfb-a895-460f2fbb3380 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:13.988774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eda3c4-9cd1-4bfb-a895-460f2fbb3380 None admin] PATH_INFO: `/v3/projects/9dbc862abb4247f69fc1a65f462d65ce` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:13.990082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eda3c4-9cd1-4bfb-a895-460f2fbb3380 None admin] RBAC: Authorizing `identity:get_project(project_id=9dbc862abb4247f69fc1a65f462d65ce)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:13.991983 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eda3c4-9cd1-4bfb-a895-460f2fbb3380 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:13.993987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 47/86] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:13 2022] GET /identity/v3/projects/9dbc862abb4247f69fc1a65f462d65ce => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:13.995029 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eda3c4-9cd1-4bfb-a895-460f2fbb3380 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:14.045859 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6cdb1-cc20-4af7-8fee-192bb30e206d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:14.047376 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6cdb1-cc20-4af7-8fee-192bb30e206d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:14.093056 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eda3c4-9cd1-4bfb-a895-460f2fbb3380 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:14.119195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:14.126813 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c9449-2236-429e-b8a0-6583c72d4900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:14.127517 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c9449-2236-429e-b8a0-6583c72d4900 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:14.127805 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c9449-2236-429e-b8a0-6583c72d4900 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:14.128084 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c9449-2236-429e-b8a0-6583c72d4900 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:14.128796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c9449-2236-429e-b8a0-6583c72d4900 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:14.139513 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c9449-2236-429e-b8a0-6583c72d4900 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:14.148567 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 40/87] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:14:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:14:14.149401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6cdb1-cc20-4af7-8fee-192bb30e206d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:14.206987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 48/88] 10.222.0.24 () {60 vars in 1132 bytes} [Fri May 27 03:14:13 2022] POST /identity/v3/users => generated 289 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:14:14.206987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c9449-2236-429e-b8a0-6583c72d4900 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:14.234095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:14.261482 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6cdb1-cc20-4af7-8fee-192bb30e206d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:14.298525 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb5f66-68d0-4101-be4f-6b4f77d87b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:14.299446 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb5f66-68d0-4101-be4f-6b4f77d87b0d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:14.299729 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb5f66-68d0-4101-be4f-6b4f77d87b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:14.300007 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb5f66-68d0-4101-be4f-6b4f77d87b0d None admin] PATH_INFO: `/v3/projects/9dbc862abb4247f69fc1a65f462d65ce` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:14.302153 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb5f66-68d0-4101-be4f-6b4f77d87b0d None admin] RBAC: Authorizing `identity:get_project(project_id=9dbc862abb4247f69fc1a65f462d65ce)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:14.303708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb5f66-68d0-4101-be4f-6b4f77d87b0d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:14.305621 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 41/89] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:14 2022] GET /identity/v3/projects/9dbc862abb4247f69fc1a65f462d65ce => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:14.306250 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb5f66-68d0-4101-be4f-6b4f77d87b0d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:14.328548 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c9449-2236-429e-b8a0-6583c72d4900 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:14.354902 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:14.367872 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b3a37-52e0-4c77-adf3-f5befcc23fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:14.368941 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b3a37-52e0-4c77-adf3-f5befcc23fdd None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:14.369229 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b3a37-52e0-4c77-adf3-f5befcc23fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:14.369512 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b3a37-52e0-4c77-adf3-f5befcc23fdd None admin] PATH_INFO: `/v3/users/3e25056b9cac4b5cb4ff28c737a1cbdd` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:14.370876 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62b3a37-52e0-4c77-adf3-f5befcc23fdd None admin] RBAC: Authorizing `identity:get_user(user_id=3e25056b9cac4b5cb4ff28c737a1cbdd)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:14.372345 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62b3a37-52e0-4c77-adf3-f5befcc23fdd None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:14.374776 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 49/90] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:14 2022] GET /identity/v3/users/3e25056b9cac4b5cb4ff28c737a1cbdd => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:14.379897 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b3a37-52e0-4c77-adf3-f5befcc23fdd None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:14.419281 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb5f66-68d0-4101-be4f-6b4f77d87b0d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:14.467628 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254195e1-8408-4f7b-969e-be70613a62db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:14.468225 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254195e1-8408-4f7b-969e-be70613a62db None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:14.468503 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254195e1-8408-4f7b-969e-be70613a62db None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:14.468774 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254195e1-8408-4f7b-969e-be70613a62db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:14.469609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254195e1-8408-4f7b-969e-be70613a62db None admin] RBAC: Adding query filter params (role.id=343337c6f25943a9ad412047b70ddefa, scope.project.id=9dbc862abb4247f69fc1a65f462d65ce, user.id=3e25056b9cac4b5cb4ff28c737a1cbdd) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:14.469923 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254195e1-8408-4f7b-969e-be70613a62db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:14.471472 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254195e1-8408-4f7b-969e-be70613a62db None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:14.479695 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 42/91] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:14:14 2022] GET /identity/v3/role_assignments?user.id=3e25056b9cac4b5cb4ff28c737a1cbdd&scope.project.id=9dbc862abb4247f69fc1a65f462d65ce&role.id=343337c6f25943a9ad412047b70ddefa => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:14.480692 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254195e1-8408-4f7b-969e-be70613a62db None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:14.488481 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:14.492771 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b3a37-52e0-4c77-adf3-f5befcc23fdd None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:14.543976 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc60f3b3-d405-43c4-94a0-b8a2eb61e813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:14.545721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60f3b3-d405-43c4-94a0-b8a2eb61e813 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:14.546005 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60f3b3-d405-43c4-94a0-b8a2eb61e813 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:14.546282 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60f3b3-d405-43c4-94a0-b8a2eb61e813 None admin] PATH_INFO: `/v3/projects/9dbc862abb4247f69fc1a65f462d65ce` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:14.547628 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc60f3b3-d405-43c4-94a0-b8a2eb61e813 None admin] RBAC: Authorizing `identity:get_project(project_id=9dbc862abb4247f69fc1a65f462d65ce)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:14.549350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc60f3b3-d405-43c4-94a0-b8a2eb61e813 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:14.551941 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 50/92] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:14 2022] GET /identity/v3/projects/9dbc862abb4247f69fc1a65f462d65ce => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:14.552626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc60f3b3-d405-43c4-94a0-b8a2eb61e813 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:14.635517 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254195e1-8408-4f7b-969e-be70613a62db None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:14.639478 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:14.677499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e627a2-f74d-497c-8f19-b4af8529b46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:14.677899 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e627a2-f74d-497c-8f19-b4af8529b46c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:14.678192 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e627a2-f74d-497c-8f19-b4af8529b46c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:14.678471 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e627a2-f74d-497c-8f19-b4af8529b46c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:14.679479 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e627a2-f74d-497c-8f19-b4af8529b46c None admin] RBAC: Adding query filter params (role.id=d7133ce917f0431883775551ab086d5d, scope.project.id=9dbc862abb4247f69fc1a65f462d65ce, user.id=3e25056b9cac4b5cb4ff28c737a1cbdd) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:14.679739 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e627a2-f74d-497c-8f19-b4af8529b46c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:14.681315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e627a2-f74d-497c-8f19-b4af8529b46c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:14.686385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc60f3b3-d405-43c4-94a0-b8a2eb61e813 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:14.688652 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 43/93] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:14:14 2022] GET /identity/v3/role_assignments?user.id=3e25056b9cac4b5cb4ff28c737a1cbdd&scope.project.id=9dbc862abb4247f69fc1a65f462d65ce&role.id=d7133ce917f0431883775551ab086d5d => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:14.689715 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e627a2-f74d-497c-8f19-b4af8529b46c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:14.751598 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45247c31-2cd8-428d-b380-3c88d089912b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:14.753051 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45247c31-2cd8-428d-b380-3c88d089912b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:14.753334 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45247c31-2cd8-428d-b380-3c88d089912b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:14.753611 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45247c31-2cd8-428d-b380-3c88d089912b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:14.824595 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:14.865734 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45247c31-2cd8-428d-b380-3c88d089912b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:14.867393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45247c31-2cd8-428d-b380-3c88d089912b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:14.884229 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e627a2-f74d-497c-8f19-b4af8529b46c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:14.973690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b21f8b-88a8-49e6-bf5d-4202899483f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:14.974455 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b21f8b-88a8-49e6-bf5d-4202899483f3 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:14.974740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b21f8b-88a8-49e6-bf5d-4202899483f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:14.975053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b21f8b-88a8-49e6-bf5d-4202899483f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:14.975894 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b21f8b-88a8-49e6-bf5d-4202899483f3 None admin] RBAC: Adding query filter params (role.id=e5a908bd3ed5445b82a6053246533b88, scope.project.id=9dbc862abb4247f69fc1a65f462d65ce, user.id=93ab1998e69940199adfdba4433a4d08) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:14.976302 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b21f8b-88a8-49e6-bf5d-4202899483f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:14.978908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b21f8b-88a8-49e6-bf5d-4202899483f3 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:14.985789 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 44/94] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:14:14 2022] GET /identity/v3/role_assignments?user.id=93ab1998e69940199adfdba4433a4d08&scope.project.id=9dbc862abb4247f69fc1a65f462d65ce&role.id=e5a908bd3ed5445b82a6053246533b88 => generated 259 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:15.054180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 51/95] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:14:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:14:15.060230 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b21f8b-88a8-49e6-bf5d-4202899483f3 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:15.114399 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:15.189538 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b21f8b-88a8-49e6-bf5d-4202899483f3 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:15.227183 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0922ff-0984-4c58-a77c-55b9b0d4459f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:15.228062 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0922ff-0984-4c58-a77c-55b9b0d4459f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:15.228346 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0922ff-0984-4c58-a77c-55b9b0d4459f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:15.228624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0922ff-0984-4c58-a77c-55b9b0d4459f None admin] PATH_INFO: `/v3/projects/ba26937020c4422ca6807b1bf79820b5` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:15.229847 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0922ff-0984-4c58-a77c-55b9b0d4459f None admin] RBAC: Authorizing `identity:get_project(project_id=ba26937020c4422ca6807b1bf79820b5)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:15.231352 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0922ff-0984-4c58-a77c-55b9b0d4459f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:15.233096 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 45/96] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:15 2022] GET /identity/v3/projects/ba26937020c4422ca6807b1bf79820b5 => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:15.239209 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45247c31-2cd8-428d-b380-3c88d089912b None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:15.352615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:15.391284 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45247c31-2cd8-428d-b380-3c88d089912b None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:15.438919 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef176f40-d2c4-41d9-bba8-e2266b437955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:15.439595 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef176f40-d2c4-41d9-bba8-e2266b437955 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:15.439884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef176f40-d2c4-41d9-bba8-e2266b437955 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:15.440163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef176f40-d2c4-41d9-bba8-e2266b437955 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:15.441006 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef176f40-d2c4-41d9-bba8-e2266b437955 None admin] RBAC: Adding query filter params (role.id=343337c6f25943a9ad412047b70ddefa, scope.project.id=ba26937020c4422ca6807b1bf79820b5, user.id=3e25056b9cac4b5cb4ff28c737a1cbdd) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:15.441326 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef176f40-d2c4-41d9-bba8-e2266b437955 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:15.442766 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef176f40-d2c4-41d9-bba8-e2266b437955 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:15.452195 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 52/97] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:14:15 2022] GET /identity/v3/role_assignments?user.id=3e25056b9cac4b5cb4ff28c737a1cbdd&scope.project.id=ba26937020c4422ca6807b1bf79820b5&role.id=343337c6f25943a9ad412047b70ddefa => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:17.191146 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcef8c-f988-4052-9272-d348405f9adf None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:17.191647 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcef8c-f988-4052-9272-d348405f9adf None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:17.191719 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffcef8c-f988-4052-9272-d348405f9adf None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:17.193065 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 46/98] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:17.198587 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada23cd-b2e6-49c2-8b70-942586c5b62d None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:17.199410 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada23cd-b2e6-49c2-8b70-942586c5b62d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:17.199907 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada23cd-b2e6-49c2-8b70-942586c5b62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:17.227837 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-5ada23cd-b2e6-49c2-8b70-942586c5b62d None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:17.229383 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5ada23cd-b2e6-49c2-8b70-942586c5b62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:17.317032 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 53/99] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:17.360638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffcef8c-f988-4052-9272-d348405f9adf None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:17.479352 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2da014-c2a6-4e77-8836-0012b4b48f50 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:17.479986 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2da014-c2a6-4e77-8836-0012b4b48f50 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:17.480495 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2da014-c2a6-4e77-8836-0012b4b48f50 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:17.482174 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 54/100] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:14:17.488751 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67524d0-7f99-4f39-b27e-a0825b72e3c2 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:17.489404 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67524d0-7f99-4f39-b27e-a0825b72e3c2 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:17.489883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67524d0-7f99-4f39-b27e-a0825b72e3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:17.507918 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:17.520140 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-d67524d0-7f99-4f39-b27e-a0825b72e3c2 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:17.520599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffcef8c-f988-4052-9272-d348405f9adf None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:17.521357 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d67524d0-7f99-4f39-b27e-a0825b72e3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:17.584788 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86838fe-561a-4268-9c43-ca90f5fa613e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:17.586071 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86838fe-561a-4268-9c43-ca90f5fa613e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:17.586673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86838fe-561a-4268-9c43-ca90f5fa613e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:17.587150 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86838fe-561a-4268-9c43-ca90f5fa613e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:17.611874 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86838fe-561a-4268-9c43-ca90f5fa613e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:17.619965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-c86838fe-561a-4268-9c43-ca90f5fa613e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:17.619965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:17.619965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:17.619965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:17.619965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:17.619965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:17.619965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:17.619965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:17.619965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:17.619965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:17.619965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:17.619965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:17.619965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:17.619965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:17.619965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:17.619965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:17.620864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:17.620864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:17.620864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:17.620864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:17.620864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:17.620864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:17.620864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:17.620864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:17.620864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:17.620864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:17.620864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:17.620864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:14:17.620864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:17.620864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:17.620864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:17.620864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:17.621619 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:17.621619 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:17.621619 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:17.621619 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:17.621619 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:17.621619 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:17.621619 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:17.621619 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:17.621619 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:17.621619 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:17.621619 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:17.621619 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:14:17.628701 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 47/101] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:17.634428 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86838fe-561a-4268-9c43-ca90f5fa613e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:17.653861 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 55/102] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:14:17.653861 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:17.692624 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67524d0-7f99-4f39-b27e-a0825b72e3c2 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:17.838868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67524d0-7f99-4f39-b27e-a0825b72e3c2 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:17.841186 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86838fe-561a-4268-9c43-ca90f5fa613e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:17.878340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5eb753-7932-4f7e-ab45-fa9754eb0616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:17.879869 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5eb753-7932-4f7e-ab45-fa9754eb0616 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:17.880814 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5eb753-7932-4f7e-ab45-fa9754eb0616 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:17.881096 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5eb753-7932-4f7e-ab45-fa9754eb0616 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:17.903681 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5eb753-7932-4f7e-ab45-fa9754eb0616 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:17.913010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca888f-0cf8-49db-8f65-2ea2d6ae6ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:17.913699 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca888f-0cf8-49db-8f65-2ea2d6ae6ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:17.913984 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca888f-0cf8-49db-8f65-2ea2d6ae6ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:17.918899 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca888f-0cf8-49db-8f65-2ea2d6ae6ab9 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:17.918899 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca888f-0cf8-49db-8f65-2ea2d6ae6ab9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:17.918899 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca888f-0cf8-49db-8f65-2ea2d6ae6ab9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:17.918899 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca888f-0cf8-49db-8f65-2ea2d6ae6ab9 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:17.920564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-ae5eb753-7932-4f7e-ab45-fa9754eb0616 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:14:17.920564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:17.920564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:17.920564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:17.920564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:17.920564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:17.920564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:17.920564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:17.920564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:17.920564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:17.920564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:17.920564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:17.920564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:17.920564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:17.920564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:17.920564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:17.921462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:17.921462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:17.921462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:17.921462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:17.921462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:17.921462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:17.921462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:17.921462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:17.921462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:17.921462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:17.921462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:17.921462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:14:17.921462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:17.921462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:17.921462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:17.921462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:17.922206 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:17.922206 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:17.922206 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:17.922206 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:17.922206 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:17.922206 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:17.922206 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:17.922206 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:17.922206 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:17.922206 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:17.922206 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:14:17.922206 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:14:17.925502 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 56/103] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:14:17.929110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da0523-f879-4d64-a7e0-fce60b0bda4c None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:17.930729 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da0523-f879-4d64-a7e0-fce60b0bda4c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:17.931160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da0523-f879-4d64-a7e0-fce60b0bda4c None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:17.936447 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 57/104] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:17.937566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da0523-f879-4d64-a7e0-fce60b0bda4c None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:17.939370 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 48/105] 10.222.0.24 () {56 vars in 1101 bytes} [Fri May 27 03:14:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:17.943002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1e286-116f-4098-b960-477c75e1f2bc None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:17.943437 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1e286-116f-4098-b960-477c75e1f2bc None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:17.944371 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1e286-116f-4098-b960-477c75e1f2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:17.979881 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-62b1e286-116f-4098-b960-477c75e1f2bc None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:17.981107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-62b1e286-116f-4098-b960-477c75e1f2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:18.126574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:18.128973 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:18.186995 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da0523-f879-4d64-a7e0-fce60b0bda4c None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:18.197567 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 49/106] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:18.199974 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1e286-116f-4098-b960-477c75e1f2bc None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:18.230382 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4913435c-feb1-4485-9134-2b32e0a65727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:18.231118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913435c-feb1-4485-9134-2b32e0a65727 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:18.231818 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913435c-feb1-4485-9134-2b32e0a65727 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:18.231818 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4913435c-feb1-4485-9134-2b32e0a65727 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:18.232577 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4913435c-feb1-4485-9134-2b32e0a65727 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:18.232886 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4913435c-feb1-4485-9134-2b32e0a65727 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:18.234376 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4913435c-feb1-4485-9134-2b32e0a65727 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:18.247213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 58/107] 10.222.0.24 () {56 vars in 1101 bytes} [Fri May 27 03:14:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:18.247213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465060b9-5130-41ed-a635-6dcb2faa445f None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:18.247213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465060b9-5130-41ed-a635-6dcb2faa445f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:18.247213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465060b9-5130-41ed-a635-6dcb2faa445f None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:18.247213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 59/108] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:18.247799 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465060b9-5130-41ed-a635-6dcb2faa445f None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:18.345960 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1e286-116f-4098-b960-477c75e1f2bc None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:18.367505 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465060b9-5130-41ed-a635-6dcb2faa445f None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:18.380536 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:18.388153 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f38bd-3ce4-46bd-bd1e-5eaf9ddd4931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:18.389566 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f38bd-3ce4-46bd-bd1e-5eaf9ddd4931 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:18.389850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f38bd-3ce4-46bd-bd1e-5eaf9ddd4931 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:18.390126 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f38bd-3ce4-46bd-bd1e-5eaf9ddd4931 None admin] PATH_INFO: `/v3/users/3e25056b9cac4b5cb4ff28c737a1cbdd` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:18.391351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f38bd-3ce4-46bd-bd1e-5eaf9ddd4931 None admin] RBAC: Authorizing `identity:get_user(user_id=3e25056b9cac4b5cb4ff28c737a1cbdd)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:18.392804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f38bd-3ce4-46bd-bd1e-5eaf9ddd4931 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:18.394438 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 50/109] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:18 2022] GET /identity/v3/users/3e25056b9cac4b5cb4ff28c737a1cbdd => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:18.399500 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f38bd-3ce4-46bd-bd1e-5eaf9ddd4931 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:18.404134 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56d621a-2483-45fd-9ab1-83590295fd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:18.404134 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d621a-2483-45fd-9ab1-83590295fd5d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:18.404134 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d621a-2483-45fd-9ab1-83590295fd5d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:18.404134 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d621a-2483-45fd-9ab1-83590295fd5d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:18.405751 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56d621a-2483-45fd-9ab1-83590295fd5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:18.407230 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56d621a-2483-45fd-9ab1-83590295fd5d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:18.408842 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 60/110] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:14:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:18.409975 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074345b-d4e4-4806-868d-7b0368ed87eb None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:18.410258 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074345b-d4e4-4806-868d-7b0368ed87eb None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:18.410535 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074345b-d4e4-4806-868d-7b0368ed87eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:18.426773 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:18.427197 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-f074345b-d4e4-4806-868d-7b0368ed87eb None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:18.428654 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f074345b-d4e4-4806-868d-7b0368ed87eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:18.503669 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 61/111] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:18.505013 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf4b09-8907-4d73-a80e-c61e0f4664fd None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:18.505305 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf4b09-8907-4d73-a80e-c61e0f4664fd None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:18.505587 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf4b09-8907-4d73-a80e-c61e0f4664fd None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:18.507015 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 62/112] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:18.507620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf4b09-8907-4d73-a80e-c61e0f4664fd None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:18.518963 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f38bd-3ce4-46bd-bd1e-5eaf9ddd4931 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:18.554503 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e2fac-57ab-4c7d-b9b8-7b35eca9a436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:18.555604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e2fac-57ab-4c7d-b9b8-7b35eca9a436 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:18.555891 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e2fac-57ab-4c7d-b9b8-7b35eca9a436 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:18.556171 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e2fac-57ab-4c7d-b9b8-7b35eca9a436 None admin] PATH_INFO: `/v3/users/f4ce8d7e0ba24e4ebab4b74a1f28995a` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:18.570065 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e2fac-57ab-4c7d-b9b8-7b35eca9a436 None admin] RBAC: Authorizing `identity:get_user(user_id=f4ce8d7e0ba24e4ebab4b74a1f28995a)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:18.571576 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e2fac-57ab-4c7d-b9b8-7b35eca9a436 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:18.573280 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 51/113] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:18 2022] GET /identity/v3/users/f4ce8d7e0ba24e4ebab4b74a1f28995a => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:18.574034 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e2fac-57ab-4c7d-b9b8-7b35eca9a436 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:18.625808 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf4b09-8907-4d73-a80e-c61e0f4664fd None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:18.637887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:18.653577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:18.662330 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860ab9d-bb09-4606-a888-69dd1af984d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:18.663315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860ab9d-bb09-4606-a888-69dd1af984d2 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:18.663617 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860ab9d-bb09-4606-a888-69dd1af984d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:18.663949 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860ab9d-bb09-4606-a888-69dd1af984d2 None admin] PATH_INFO: `/v3/projects/9dbc862abb4247f69fc1a65f462d65ce` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:18.665143 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8860ab9d-bb09-4606-a888-69dd1af984d2 None admin] RBAC: Authorizing `identity:get_project(project_id=9dbc862abb4247f69fc1a65f462d65ce)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:18.666589 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8860ab9d-bb09-4606-a888-69dd1af984d2 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:18.672955 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 63/114] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:18 2022] GET /identity/v3/projects/9dbc862abb4247f69fc1a65f462d65ce => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:18.674168 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad672c0-9ff1-4989-9470-f921c80fefc8 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:18.674457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad672c0-9ff1-4989-9470-f921c80fefc8 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:18.674738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad672c0-9ff1-4989-9470-f921c80fefc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:18.675277 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e2fac-57ab-4c7d-b9b8-7b35eca9a436 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:18.691258 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-9ad672c0-9ff1-4989-9470-f921c80fefc8 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:18.692499 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-9ad672c0-9ff1-4989-9470-f921c80fefc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:18.709702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f8e3a-e89b-40da-9210-1d7a40e51095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:18.710371 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f8e3a-e89b-40da-9210-1d7a40e51095 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:18.710658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f8e3a-e89b-40da-9210-1d7a40e51095 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:18.710964 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f8e3a-e89b-40da-9210-1d7a40e51095 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:18.711676 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f8e3a-e89b-40da-9210-1d7a40e51095 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:18.713147 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f8e3a-e89b-40da-9210-1d7a40e51095 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:18.766202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 64/115] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:18.767095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad672c0-9ff1-4989-9470-f921c80fefc8 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:18.809471 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 52/116] 10.222.0.24 () {60 vars in 1141 bytes} [Fri May 27 03:14:18 2022] POST /identity/v3/projects => generated 302 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:14:18.811338 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f8e3a-e89b-40da-9210-1d7a40e51095 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:18.888509 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad672c0-9ff1-4989-9470-f921c80fefc8 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:18.906393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:18.930172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f8e3a-e89b-40da-9210-1d7a40e51095 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:18.937225 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01582c97-0926-4ec1-8865-1b73c253ce89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:18.938066 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01582c97-0926-4ec1-8865-1b73c253ce89 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:18.938401 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01582c97-0926-4ec1-8865-1b73c253ce89 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:18.938646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01582c97-0926-4ec1-8865-1b73c253ce89 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:18.955398 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01582c97-0926-4ec1-8865-1b73c253ce89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:18.963971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-01582c97-0926-4ec1-8865-1b73c253ce89 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:14:18.963971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:18.963971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:18.963971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:18.963971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:18.963971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:18.963971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:18.963971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:18.963971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:18.963971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:18.963971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:18.963971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:18.963971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:18.963971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:18.963971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:18.963971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:18.965118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:18.965118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:18.965118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:18.965118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:18.965118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:18.965118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:18.965118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:18.965118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:18.965118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:18.965118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:18.965118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:18.965118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:14:18.965118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:18.965118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:18.965118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:18.965118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:18.966038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:18.966038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:18.966038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:18.966038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:18.966038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:18.966038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:18.966038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:18.966038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:18.966038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:18.966038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:18.966038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:14:18.966038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:14:18.966038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 65/117] 10.222.0.24 () {56 vars in 1114 bytes} [Fri May 27 03:14:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:18.966038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01582c97-0926-4ec1-8865-1b73c253ce89 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:18.977635 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:18.985600 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1514135-bdac-448d-b811-4aa9920a31de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:18.986391 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1514135-bdac-448d-b811-4aa9920a31de None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:18.986673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1514135-bdac-448d-b811-4aa9920a31de None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:18.987098 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1514135-bdac-448d-b811-4aa9920a31de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:19.068840 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1514135-bdac-448d-b811-4aa9920a31de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:19.070659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1514135-bdac-448d-b811-4aa9920a31de None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:19.087035 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01582c97-0926-4ec1-8865-1b73c253ce89 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:19.125556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16ecef-8649-4616-892e-86f40a48af59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:19.126767 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16ecef-8649-4616-892e-86f40a48af59 None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:19.127253 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16ecef-8649-4616-892e-86f40a48af59 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:19.127781 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16ecef-8649-4616-892e-86f40a48af59 None admin] PATH_INFO: `/v3/projects/9dbc862abb4247f69fc1a65f462d65ce/users/3e25056b9cac4b5cb4ff28c737a1cbdd/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:19.130078 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c16ecef-8649-4616-892e-86f40a48af59 None admin] RBAC: Authorizing `identity:create_grant(project_id=9dbc862abb4247f69fc1a65f462d65ce, user_id=3e25056b9cac4b5cb4ff28c737a1cbdd, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:19.131900 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c16ecef-8649-4616-892e-86f40a48af59 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:19.171702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 53/118] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:14:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:14:19.175283 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1514135-bdac-448d-b811-4aa9920a31de None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:19.206008 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:19.215374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 66/119] 10.222.0.24 () {58 vars in 1439 bytes} [Fri May 27 03:14:18 2022] PUT /identity/v3/projects/9dbc862abb4247f69fc1a65f462d65ce/users/3e25056b9cac4b5cb4ff28c737a1cbdd/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:14:19.215374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e081c17-859b-42dc-8746-4ef96fb63a95 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:19.215374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e081c17-859b-42dc-8746-4ef96fb63a95 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:19.215374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e081c17-859b-42dc-8746-4ef96fb63a95 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:19.215374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 67/120] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:19.215374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e081c17-859b-42dc-8746-4ef96fb63a95 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:19.255378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:19.293999 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1514135-bdac-448d-b811-4aa9920a31de None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:19.343699 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cce63b-5f0f-4064-958a-713d90858be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:19.344633 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cce63b-5f0f-4064-958a-713d90858be5 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:19.344984 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cce63b-5f0f-4064-958a-713d90858be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:19.345356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cce63b-5f0f-4064-958a-713d90858be5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:19.357977 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cce63b-5f0f-4064-958a-713d90858be5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:19.365263 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e081c17-859b-42dc-8746-4ef96fb63a95 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:19.366081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-07cce63b-5f0f-4064-958a-713d90858be5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:14:19.366081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:19.366081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:19.366081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:19.366081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:19.366081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:19.366081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:19.366081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:19.366081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:19.366081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:19.366081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:19.366081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:19.366081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:19.366081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:19.366081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:19.366081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:19.366930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:19.366930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:19.366930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:19.366930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:19.366930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:19.366930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:19.366930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:19.366930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:19.366930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:19.366930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:19.366930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:19.366930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:14:19.366930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:19.366930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:19.366930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:19.366930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:19.367702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:19.367702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:19.367702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:19.367702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:19.367702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:19.367702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:19.367702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:19.367702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:19.367702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:19.367702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:19.367702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:14:19.367702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:14:19.367702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 54/121] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:14:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:19.368322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cce63b-5f0f-4064-958a-713d90858be5 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:19.416010 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd5269-f8e6-4002-a2e6-2a9f6f38effa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:19.416796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd5269-f8e6-4002-a2e6-2a9f6f38effa None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:19.417152 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd5269-f8e6-4002-a2e6-2a9f6f38effa None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:19.417501 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd5269-f8e6-4002-a2e6-2a9f6f38effa None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:19.418623 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd5269-f8e6-4002-a2e6-2a9f6f38effa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:19.419030 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd5269-f8e6-4002-a2e6-2a9f6f38effa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:19.422169 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd5269-f8e6-4002-a2e6-2a9f6f38effa None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:19.434289 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 68/122] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:14:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:19.436300 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729aa94-cb34-4ec3-95aa-8329799e9f39 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:19.436589 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729aa94-cb34-4ec3-95aa-8329799e9f39 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:19.436865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729aa94-cb34-4ec3-95aa-8329799e9f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:19.456332 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-5729aa94-cb34-4ec3-95aa-8329799e9f39 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:19.457526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5729aa94-cb34-4ec3-95aa-8329799e9f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:19.487663 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:19.498235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cce63b-5f0f-4064-958a-713d90858be5 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:19.539690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f54ddd5-3fd6-4eac-8b3d-c229e0d5c0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:19.540782 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f54ddd5-3fd6-4eac-8b3d-c229e0d5c0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:19.541221 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f54ddd5-3fd6-4eac-8b3d-c229e0d5c0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:19.541641 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f54ddd5-3fd6-4eac-8b3d-c229e0d5c0c7 None admin] PATH_INFO: `/v3/projects/9dbc862abb4247f69fc1a65f462d65ce` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:19.542196 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 69/123] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:19.543271 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729aa94-cb34-4ec3-95aa-8329799e9f39 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:19.543552 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f54ddd5-3fd6-4eac-8b3d-c229e0d5c0c7 None admin] RBAC: Authorizing `identity:get_project(project_id=9dbc862abb4247f69fc1a65f462d65ce)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:19.545816 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f54ddd5-3fd6-4eac-8b3d-c229e0d5c0c7 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:19.548719 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 55/124] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:19 2022] GET /identity/v3/projects/9dbc862abb4247f69fc1a65f462d65ce => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:19.554861 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f54ddd5-3fd6-4eac-8b3d-c229e0d5c0c7 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:19.573975 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:19.667925 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f54ddd5-3fd6-4eac-8b3d-c229e0d5c0c7 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:19.690078 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729aa94-cb34-4ec3-95aa-8329799e9f39 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:19.705381 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25877d9e-03e8-4c1a-b3b1-c44c014418b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:19.706254 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25877d9e-03e8-4c1a-b3b1-c44c014418b7 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:19.706534 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25877d9e-03e8-4c1a-b3b1-c44c014418b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:19.706803 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25877d9e-03e8-4c1a-b3b1-c44c014418b7 None admin] PATH_INFO: `/v3/users/3e25056b9cac4b5cb4ff28c737a1cbdd` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:19.708051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25877d9e-03e8-4c1a-b3b1-c44c014418b7 None admin] RBAC: Authorizing `identity:get_user(user_id=3e25056b9cac4b5cb4ff28c737a1cbdd)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:19.709496 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25877d9e-03e8-4c1a-b3b1-c44c014418b7 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:19.711377 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 56/125] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:19 2022] GET /identity/v3/users/3e25056b9cac4b5cb4ff28c737a1cbdd => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:19.711923 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25877d9e-03e8-4c1a-b3b1-c44c014418b7 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:19.736450 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a42c67-b1b8-4f56-b95d-71a78ed50feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:19.737117 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a42c67-b1b8-4f56-b95d-71a78ed50feb None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:19.737397 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a42c67-b1b8-4f56-b95d-71a78ed50feb None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:19.737668 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a42c67-b1b8-4f56-b95d-71a78ed50feb None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:19.738422 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a42c67-b1b8-4f56-b95d-71a78ed50feb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:19.738726 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a42c67-b1b8-4f56-b95d-71a78ed50feb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:19.740127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a42c67-b1b8-4f56-b95d-71a78ed50feb None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:19.748735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 70/126] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:19.749538 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a42c67-b1b8-4f56-b95d-71a78ed50feb None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:19.751224 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:19.845178 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:19.847794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25877d9e-03e8-4c1a-b3b1-c44c014418b7 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:19.868718 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a42c67-b1b8-4f56-b95d-71a78ed50feb None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:19.893832 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbc0ee-70ac-439f-bbc3-2f6d1b13fcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:19.895234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbc0ee-70ac-439f-bbc3-2f6d1b13fcaa None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:19.895528 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbc0ee-70ac-439f-bbc3-2f6d1b13fcaa None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:19.895813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcbc0ee-70ac-439f-bbc3-2f6d1b13fcaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:19.896918 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcbc0ee-70ac-439f-bbc3-2f6d1b13fcaa None admin] RBAC: Adding query filter params (role.id=aa0d3c6b36a24a718f88d0f4309d26d9, scope.project.id=9dbc862abb4247f69fc1a65f462d65ce, user.id=f4ce8d7e0ba24e4ebab4b74a1f28995a) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:19.897658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcbc0ee-70ac-439f-bbc3-2f6d1b13fcaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:19.899439 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcbc0ee-70ac-439f-bbc3-2f6d1b13fcaa None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:19.905994 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde157e-16b4-43c9-b872-beb39b1b0da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:19.907005 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde157e-16b4-43c9-b872-beb39b1b0da0 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:19.907442 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 57/127] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:14:19 2022] GET /identity/v3/role_assignments?user.id=f4ce8d7e0ba24e4ebab4b74a1f28995a&scope.project.id=9dbc862abb4247f69fc1a65f462d65ce&role.id=aa0d3c6b36a24a718f88d0f4309d26d9 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:19.907560 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde157e-16b4-43c9-b872-beb39b1b0da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:19.907821 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde157e-16b4-43c9-b872-beb39b1b0da0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:19.908043 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbc0ee-70ac-439f-bbc3-2f6d1b13fcaa None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:19.920098 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde157e-16b4-43c9-b872-beb39b1b0da0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:19.928527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-dcde157e-16b4-43c9-b872-beb39b1b0da0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:19.928527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:19.928527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:19.928527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:19.928527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:19.928527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:19.928527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:19.928527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:19.928527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:19.928527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:19.928527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:19.928527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:19.928527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:19.928527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:19.928527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:19.928527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:19.929786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:19.929786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:19.929786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:19.929786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:19.929786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:19.929786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:19.929786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:19.929786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:19.929786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:19.929786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:19.929786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:19.929786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:14:19.929786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:19.929786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:19.929786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:19.929786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:19.931445 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:19.931445 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:19.931445 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:19.931445 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:19.931445 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:19.931445 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:19.931445 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:19.931445 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:19.931445 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:19.931445 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:19.931445 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:19.931445 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:14:19.932914 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 71/128] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:19.937943 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde157e-16b4-43c9-b872-beb39b1b0da0 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:20.041580 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:20.054585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcbc0ee-70ac-439f-bbc3-2f6d1b13fcaa None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:20.109422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb97425-d7c1-4472-82ac-f4bdf5055c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:20.110310 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb97425-d7c1-4472-82ac-f4bdf5055c67 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:20.110594 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb97425-d7c1-4472-82ac-f4bdf5055c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:20.110889 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb97425-d7c1-4472-82ac-f4bdf5055c67 None admin] PATH_INFO: `/v3/projects/9dbc862abb4247f69fc1a65f462d65ce` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:20.112772 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb97425-d7c1-4472-82ac-f4bdf5055c67 None admin] RBAC: Authorizing `identity:get_project(project_id=9dbc862abb4247f69fc1a65f462d65ce)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:20.114300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb97425-d7c1-4472-82ac-f4bdf5055c67 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:20.116372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 58/129] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:19 2022] GET /identity/v3/projects/9dbc862abb4247f69fc1a65f462d65ce => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:20.116942 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb97425-d7c1-4472-82ac-f4bdf5055c67 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:20.193757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde157e-16b4-43c9-b872-beb39b1b0da0 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:20.244974 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb97425-d7c1-4472-82ac-f4bdf5055c67 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:20.273098 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15419ca-bdcd-419a-a3bb-921f094d70f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:20.274137 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15419ca-bdcd-419a-a3bb-921f094d70f8 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:20.274424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15419ca-bdcd-419a-a3bb-921f094d70f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:20.274703 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15419ca-bdcd-419a-a3bb-921f094d70f8 None admin] PATH_INFO: `/v3/users/93ab1998e69940199adfdba4433a4d08` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:20.279840 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15419ca-bdcd-419a-a3bb-921f094d70f8 None admin] RBAC: Authorizing `identity:get_user(user_id=93ab1998e69940199adfdba4433a4d08)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:20.281110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c9ab9-60ce-4dbb-98e6-472bdffdebe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:20.281331 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15419ca-bdcd-419a-a3bb-921f094d70f8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:20.281787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c9ab9-60ce-4dbb-98e6-472bdffdebe3 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:20.282076 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c9ab9-60ce-4dbb-98e6-472bdffdebe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:20.282355 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c9ab9-60ce-4dbb-98e6-472bdffdebe3 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:20.283163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 72/130] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:19 2022] GET /identity/v3/users/93ab1998e69940199adfdba4433a4d08 => generated 247 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:20.283324 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c9ab9-60ce-4dbb-98e6-472bdffdebe3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:20.283466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c9ab9-60ce-4dbb-98e6-472bdffdebe3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:20.283696 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15419ca-bdcd-419a-a3bb-921f094d70f8 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:20.284963 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c9ab9-60ce-4dbb-98e6-472bdffdebe3 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:20.299136 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:20.302325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 59/131] 10.222.0.24 () {56 vars in 1101 bytes} [Fri May 27 03:14:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:20.303013 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c9ab9-60ce-4dbb-98e6-472bdffdebe3 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:20.344479 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:20.416674 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15419ca-bdcd-419a-a3bb-921f094d70f8 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:20.457245 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6aecae-5888-4239-886b-4647bc2b97fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:20.458174 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6aecae-5888-4239-886b-4647bc2b97fa None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:20.458587 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6aecae-5888-4239-886b-4647bc2b97fa None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:20.458886 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6aecae-5888-4239-886b-4647bc2b97fa None admin] PATH_INFO: `/v3/projects/9dbc862abb4247f69fc1a65f462d65ce/users/3e25056b9cac4b5cb4ff28c737a1cbdd/roles/d7133ce917f0431883775551ab086d5d` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:20.461296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6aecae-5888-4239-886b-4647bc2b97fa None admin] RBAC: Authorizing `identity:create_grant(project_id=9dbc862abb4247f69fc1a65f462d65ce, user_id=3e25056b9cac4b5cb4ff28c737a1cbdd, role_id=d7133ce917f0431883775551ab086d5d)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:20.463179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6aecae-5888-4239-886b-4647bc2b97fa None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:20.475548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c9ab9-60ce-4dbb-98e6-472bdffdebe3 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:20.513329 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d68fb8-4bdb-4db0-997b-ad1a46f8f7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:20.514747 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d68fb8-4bdb-4db0-997b-ad1a46f8f7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:20.515062 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d68fb8-4bdb-4db0-997b-ad1a46f8f7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:20.515397 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d68fb8-4bdb-4db0-997b-ad1a46f8f7e2 None admin] PATH_INFO: `/v3/projects/9dbc862abb4247f69fc1a65f462d65ce` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:20.516692 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d68fb8-4bdb-4db0-997b-ad1a46f8f7e2 None admin] RBAC: Authorizing `identity:get_project(project_id=9dbc862abb4247f69fc1a65f462d65ce)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:20.518199 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d68fb8-4bdb-4db0-997b-ad1a46f8f7e2 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:20.520311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 60/132] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:20 2022] GET /identity/v3/projects/9dbc862abb4247f69fc1a65f462d65ce => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:20.521190 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d68fb8-4bdb-4db0-997b-ad1a46f8f7e2 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:20.542159 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 73/133] 10.222.0.24 () {58 vars in 1439 bytes} [Fri May 27 03:14:20 2022] PUT /identity/v3/projects/9dbc862abb4247f69fc1a65f462d65ce/users/3e25056b9cac4b5cb4ff28c737a1cbdd/roles/d7133ce917f0431883775551ab086d5d => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:14:20.542949 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6aecae-5888-4239-886b-4647bc2b97fa None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:20.668548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:20.682665 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d68fb8-4bdb-4db0-997b-ad1a46f8f7e2 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:20.685886 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:20.720606 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab68956-e22e-4265-8212-1491e6ea5609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:20.721580 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab68956-e22e-4265-8212-1491e6ea5609 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:20.721835 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab68956-e22e-4265-8212-1491e6ea5609 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:20.722118 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab68956-e22e-4265-8212-1491e6ea5609 None admin] PATH_INFO: `/v3/users/3e25056b9cac4b5cb4ff28c737a1cbdd` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:20.723514 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab68956-e22e-4265-8212-1491e6ea5609 None admin] RBAC: Authorizing `identity:get_user(user_id=3e25056b9cac4b5cb4ff28c737a1cbdd)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:20.724989 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab68956-e22e-4265-8212-1491e6ea5609 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:20.728129 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 61/134] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:20 2022] GET /identity/v3/users/3e25056b9cac4b5cb4ff28c737a1cbdd => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:20.732825 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab68956-e22e-4265-8212-1491e6ea5609 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:20.736799 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6aecae-5888-4239-886b-4647bc2b97fa None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:20.799602 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907435d-b200-4b28-8d7c-eced9932095f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:20.800933 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907435d-b200-4b28-8d7c-eced9932095f None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:20.801421 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907435d-b200-4b28-8d7c-eced9932095f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:20.801893 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907435d-b200-4b28-8d7c-eced9932095f None admin] PATH_INFO: `/v3/projects/9dbc862abb4247f69fc1a65f462d65ce/users/93ab1998e69940199adfdba4433a4d08/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:20.804213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0907435d-b200-4b28-8d7c-eced9932095f None admin] RBAC: Authorizing `identity:create_grant(project_id=9dbc862abb4247f69fc1a65f462d65ce, user_id=93ab1998e69940199adfdba4433a4d08, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:20.805919 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0907435d-b200-4b28-8d7c-eced9932095f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:20.839131 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab68956-e22e-4265-8212-1491e6ea5609 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:20.876904 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db684774-0c4c-4118-952b-442f0ff8d1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:20.877775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db684774-0c4c-4118-952b-442f0ff8d1b8 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:20.878109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db684774-0c4c-4118-952b-442f0ff8d1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:20.878388 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db684774-0c4c-4118-952b-442f0ff8d1b8 None admin] PATH_INFO: `/v3/projects/ba26937020c4422ca6807b1bf79820b5` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:20.879605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db684774-0c4c-4118-952b-442f0ff8d1b8 None admin] RBAC: Authorizing `identity:get_project(project_id=ba26937020c4422ca6807b1bf79820b5)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:20.881272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db684774-0c4c-4118-952b-442f0ff8d1b8 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:20.883153 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 62/135] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:20 2022] GET /identity/v3/projects/ba26937020c4422ca6807b1bf79820b5 => generated 312 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:20.889850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db684774-0c4c-4118-952b-442f0ff8d1b8 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:20.898965 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 74/136] 10.222.0.24 () {58 vars in 1439 bytes} [Fri May 27 03:14:20 2022] PUT /identity/v3/projects/9dbc862abb4247f69fc1a65f462d65ce/users/93ab1998e69940199adfdba4433a4d08/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:14:20.939549 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:21.015201 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db684774-0c4c-4118-952b-442f0ff8d1b8 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:21.063040 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f7b78-0857-420d-9591-45b09c1aab32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:21.063966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f7b78-0857-420d-9591-45b09c1aab32 None admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:21.064246 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f7b78-0857-420d-9591-45b09c1aab32 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:21.064524 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f7b78-0857-420d-9591-45b09c1aab32 None admin] PATH_INFO: `/v3/projects/ba26937020c4422ca6807b1bf79820b5/users/3e25056b9cac4b5cb4ff28c737a1cbdd/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:21.066729 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f7b78-0857-420d-9591-45b09c1aab32 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba26937020c4422ca6807b1bf79820b5, user_id=3e25056b9cac4b5cb4ff28c737a1cbdd, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:21.068303 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f7b78-0857-420d-9591-45b09c1aab32 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:21.151926 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 63/137] 10.222.0.24 () {58 vars in 1439 bytes} [Fri May 27 03:14:20 2022] PUT /identity/v3/projects/ba26937020c4422ca6807b1bf79820b5/users/3e25056b9cac4b5cb4ff28c737a1cbdd/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:14:21.604115 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7bed0-617a-4501-b9f8-a25cc6d11e57 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:21.605020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7bed0-617a-4501-b9f8-a25cc6d11e57 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:21.605477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7bed0-617a-4501-b9f8-a25cc6d11e57 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:21.607911 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 75/138] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:21.614074 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7116f74-d2d7-40c8-a468-7f50b2568d91 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:21.614976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7116f74-d2d7-40c8-a468-7f50b2568d91 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:21.615470 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7116f74-d2d7-40c8-a468-7f50b2568d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:21.633493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-b7116f74-d2d7-40c8-a468-7f50b2568d91 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:21.635133 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b7116f74-d2d7-40c8-a468-7f50b2568d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:21.711321 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 64/139] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:14:21.736868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7bed0-617a-4501-b9f8-a25cc6d11e57 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:21.839325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186076d8-c73c-4f16-9393-2eff5f972ad9 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:21.839636 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186076d8-c73c-4f16-9393-2eff5f972ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:21.839895 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186076d8-c73c-4f16-9393-2eff5f972ad9 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:21.841274 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 65/140] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:14:21.846866 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23290b-06aa-477c-aa9b-54d992dda174 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:21.847175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23290b-06aa-477c-aa9b-54d992dda174 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:21.847453 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23290b-06aa-477c-aa9b-54d992dda174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:21.865811 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-1c23290b-06aa-477c-aa9b-54d992dda174 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:21.868196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1c23290b-06aa-477c-aa9b-54d992dda174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:21.879113 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7bed0-617a-4501-b9f8-a25cc6d11e57 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:21.887592 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:21.927571 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:21.938061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941220e-0d40-4d0c-8192-06ff553ec216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:21.938880 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941220e-0d40-4d0c-8192-06ff553ec216 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:21.939167 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941220e-0d40-4d0c-8192-06ff553ec216 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:21.939443 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941220e-0d40-4d0c-8192-06ff553ec216 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:21.942151 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941220e-0d40-4d0c-8192-06ff553ec216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:21.943635 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941220e-0d40-4d0c-8192-06ff553ec216 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:21.945274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 76/141] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:14:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:21.952733 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941220e-0d40-4d0c-8192-06ff553ec216 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:21.996211 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 66/142] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:14:22.049308 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23290b-06aa-477c-aa9b-54d992dda174 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:22.086344 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941220e-0d40-4d0c-8192-06ff553ec216 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:22.158312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b07ab67-6932-4d37-b8cb-ac1f63acbcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:22.158944 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07ab67-6932-4d37-b8cb-ac1f63acbcfe None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:22.159232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07ab67-6932-4d37-b8cb-ac1f63acbcfe None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:22.159512 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07ab67-6932-4d37-b8cb-ac1f63acbcfe None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:22.160252 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b07ab67-6932-4d37-b8cb-ac1f63acbcfe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:22.161761 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b07ab67-6932-4d37-b8cb-ac1f63acbcfe None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:22.271715 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23290b-06aa-477c-aa9b-54d992dda174 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:22.287939 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 77/143] 10.222.0.24 () {60 vars in 1132 bytes} [Fri May 27 03:14:21 2022] POST /identity/v3/users => generated 283 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:14:22.316846 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae187427-6006-41e5-b28f-645aa80b791b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:22.317719 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae187427-6006-41e5-b28f-645aa80b791b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:22.318065 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae187427-6006-41e5-b28f-645aa80b791b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:22.318348 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae187427-6006-41e5-b28f-645aa80b791b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:22.330792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae187427-6006-41e5-b28f-645aa80b791b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:22.339290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-ae187427-6006-41e5-b28f-645aa80b791b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:22.339290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:22.339290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:22.339290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:22.339290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:22.339290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:22.339290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:22.339290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:22.339290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:22.339290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:22.339290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:22.339290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:22.339290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:22.339290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:22.339290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:22.339290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:22.340167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:22.340167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:22.340167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:22.340167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:22.340167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:22.340167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:22.340167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:22.340167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:22.340167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:22.340167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:22.340167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:22.340167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:14:22.340167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:22.340167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:22.340167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:22.340167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:22.341031 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:22.341031 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:22.341031 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:22.341031 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:22.341031 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:22.341031 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:22.341031 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:22.341031 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:22.341031 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:22.341031 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:22.341031 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:22.341031 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:14:22.341031 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 67/144] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:14:22.348136 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b07ab67-6932-4d37-b8cb-ac1f63acbcfe None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:22.416700 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:22.465428 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b07ab67-6932-4d37-b8cb-ac1f63acbcfe None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:22.501791 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8bdfe-7a5b-4027-87d3-7335f4565854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:22.502589 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8bdfe-7a5b-4027-87d3-7335f4565854 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:22.502902 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8bdfe-7a5b-4027-87d3-7335f4565854 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:22.503189 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8bdfe-7a5b-4027-87d3-7335f4565854 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:22.504005 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c8bdfe-7a5b-4027-87d3-7335f4565854 None admin] RBAC: Adding query filter params (name=member) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:22.504966 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c8bdfe-7a5b-4027-87d3-7335f4565854 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:22.506370 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c8bdfe-7a5b-4027-87d3-7335f4565854 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:22.516047 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 78/145] 10.222.0.24 () {56 vars in 1101 bytes} [Fri May 27 03:14:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:22.521877 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae187427-6006-41e5-b28f-645aa80b791b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:22.527182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:22.651155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae187427-6006-41e5-b28f-645aa80b791b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:22.688717 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b60866-8af1-4578-9fc9-5ff8bcf518dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:22.689558 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b60866-8af1-4578-9fc9-5ff8bcf518dd None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:22.689842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b60866-8af1-4578-9fc9-5ff8bcf518dd None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:22.690116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b60866-8af1-4578-9fc9-5ff8bcf518dd None admin] PATH_INFO: `/v3/users/3e25056b9cac4b5cb4ff28c737a1cbdd` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:22.691542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b60866-8af1-4578-9fc9-5ff8bcf518dd None admin] RBAC: Authorizing `identity:get_user(user_id=3e25056b9cac4b5cb4ff28c737a1cbdd)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:22.693080 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b60866-8af1-4578-9fc9-5ff8bcf518dd None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:22.695049 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 68/146] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:22 2022] GET /identity/v3/users/3e25056b9cac4b5cb4ff28c737a1cbdd => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:22.700415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8bdfe-7a5b-4027-87d3-7335f4565854 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:22.807432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8bdfe-7a5b-4027-87d3-7335f4565854 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:22.832719 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:22.843828 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5523f2e-262f-414c-b1fe-9dcaffa0ff8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:22.844484 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5523f2e-262f-414c-b1fe-9dcaffa0ff8d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:22.844756 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5523f2e-262f-414c-b1fe-9dcaffa0ff8d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:22.845033 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5523f2e-262f-414c-b1fe-9dcaffa0ff8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:22.914641 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5523f2e-262f-414c-b1fe-9dcaffa0ff8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:22.916245 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5523f2e-262f-414c-b1fe-9dcaffa0ff8d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:23.018940 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 79/147] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:14:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:14:23.024359 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b60866-8af1-4578-9fc9-5ff8bcf518dd None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:23.116639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:23.159449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b60866-8af1-4578-9fc9-5ff8bcf518dd None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:23.208497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69c72e2-804d-4ec3-8bbb-30308524b681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:23.209380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c72e2-804d-4ec3-8bbb-30308524b681 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:23.209662 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c72e2-804d-4ec3-8bbb-30308524b681 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:23.209935 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69c72e2-804d-4ec3-8bbb-30308524b681 None admin] PATH_INFO: `/v3/projects/9dbc862abb4247f69fc1a65f462d65ce` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:23.211174 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69c72e2-804d-4ec3-8bbb-30308524b681 None admin] RBAC: Authorizing `identity:get_project(project_id=9dbc862abb4247f69fc1a65f462d65ce)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:23.212712 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69c72e2-804d-4ec3-8bbb-30308524b681 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:23.214484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 69/148] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:23 2022] GET /identity/v3/projects/9dbc862abb4247f69fc1a65f462d65ce => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:23.220318 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5523f2e-262f-414c-b1fe-9dcaffa0ff8d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:23.249982 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:23.320307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338ab2d-d03a-45d3-9188-763d2d31ee83 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:23.320951 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338ab2d-d03a-45d3-9188-763d2d31ee83 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:23.321441 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338ab2d-d03a-45d3-9188-763d2d31ee83 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:23.323951 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 70/149] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:23.330292 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad25bd-3077-47e2-aee4-7b353ab76e27 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:23.330860 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad25bd-3077-47e2-aee4-7b353ab76e27 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:23.331335 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad25bd-3077-47e2-aee4-7b353ab76e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:23.340883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5523f2e-262f-414c-b1fe-9dcaffa0ff8d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:23.365206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-daad25bd-3077-47e2-aee4-7b353ab76e27 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:23.366756 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-daad25bd-3077-47e2-aee4-7b353ab76e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:23.383058 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37266b82-d797-4010-a2c2-f75468b1f486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:23.383729 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37266b82-d797-4010-a2c2-f75468b1f486 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:23.384012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37266b82-d797-4010-a2c2-f75468b1f486 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:23.384285 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37266b82-d797-4010-a2c2-f75468b1f486 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:23.385130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37266b82-d797-4010-a2c2-f75468b1f486 None admin] RBAC: Adding query filter params (role.id=343337c6f25943a9ad412047b70ddefa, scope.project.id=9dbc862abb4247f69fc1a65f462d65ce, user.id=3e25056b9cac4b5cb4ff28c737a1cbdd) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:23.385438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37266b82-d797-4010-a2c2-f75468b1f486 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:23.388387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37266b82-d797-4010-a2c2-f75468b1f486 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:23.400116 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 80/150] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:14:23 2022] GET /identity/v3/role_assignments?user.id=3e25056b9cac4b5cb4ff28c737a1cbdd&scope.project.id=9dbc862abb4247f69fc1a65f462d65ce&role.id=343337c6f25943a9ad412047b70ddefa => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:23.534974 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 71/151] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:23.579257 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37266b82-d797-4010-a2c2-f75468b1f486 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:23.711315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:23.738041 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37266b82-d797-4010-a2c2-f75468b1f486 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:23.799628 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7456e20-7f43-49a5-9ab7-66cfd566efc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:23.800553 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7456e20-7f43-49a5-9ab7-66cfd566efc5 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:23.800834 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7456e20-7f43-49a5-9ab7-66cfd566efc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:23.801105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7456e20-7f43-49a5-9ab7-66cfd566efc5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:23.817291 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7456e20-7f43-49a5-9ab7-66cfd566efc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:23.833338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-b7456e20-7f43-49a5-9ab7-66cfd566efc5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:14:23.833338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:23.833338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:23.833338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:23.833338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:23.833338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:23.833338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:23.833338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:23.833338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:23.833338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:23.833338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:23.833338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:23.833338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:23.833338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:23.833338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:23.833338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:23.834201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:23.834201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:23.834201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:23.834201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:23.834201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:23.834201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:23.834201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:23.834201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:23.834201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:23.834201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:23.834201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:23.834201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:14:23.834201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:23.834201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:23.834201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:23.834201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:23.834960 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:23.834960 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:23.834960 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:23.834960 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:23.834960 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:23.834960 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:23.834960 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:23.834960 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:23.834960 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:23.834960 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:23.834960 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:14:23.834960 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:14:23.834960 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 81/152] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:23.843976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad25bd-3077-47e2-aee4-7b353ab76e27 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:23.979975 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:24.021793 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad25bd-3077-47e2-aee4-7b353ab76e27 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:24.079617 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3072a-08e5-4016-9a6b-889bd9fc900b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:24.080709 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3072a-08e5-4016-9a6b-889bd9fc900b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:24.080999 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3072a-08e5-4016-9a6b-889bd9fc900b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:24.081278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3072a-08e5-4016-9a6b-889bd9fc900b None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:24.082650 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc3072a-08e5-4016-9a6b-889bd9fc900b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:24.082981 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc3072a-08e5-4016-9a6b-889bd9fc900b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:24.084451 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc3072a-08e5-4016-9a6b-889bd9fc900b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:24.094617 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 72/153] 10.222.0.24 () {56 vars in 1101 bytes} [Fri May 27 03:14:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:24.103942 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7456e20-7f43-49a5-9ab7-66cfd566efc5 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:24.219121 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb9686-a98c-4c16-984e-538638e4996f None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:24.219424 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb9686-a98c-4c16-984e-538638e4996f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:24.219682 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb9686-a98c-4c16-984e-538638e4996f None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:24.221038 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 73/154] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:24.227481 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1bc57-03c8-4a43-baca-ee7123430579 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:24.227763 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1bc57-03c8-4a43-baca-ee7123430579 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:24.228043 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1bc57-03c8-4a43-baca-ee7123430579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:24.250043 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-cff1bc57-03c8-4a43-baca-ee7123430579 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:24.251560 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-cff1bc57-03c8-4a43-baca-ee7123430579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:24.271591 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7456e20-7f43-49a5-9ab7-66cfd566efc5 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:24.340426 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92ea8d-0d23-4592-952f-fa9ac4a61593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:24.342225 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92ea8d-0d23-4592-952f-fa9ac4a61593 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:24.342514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92ea8d-0d23-4592-952f-fa9ac4a61593 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:24.342801 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92ea8d-0d23-4592-952f-fa9ac4a61593 None admin] PATH_INFO: `/v3/users/f4ce8d7e0ba24e4ebab4b74a1f28995a` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:24.347248 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92ea8d-0d23-4592-952f-fa9ac4a61593 None admin] RBAC: Authorizing `identity:get_user(user_id=f4ce8d7e0ba24e4ebab4b74a1f28995a)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:24.348940 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92ea8d-0d23-4592-952f-fa9ac4a61593 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:24.351051 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 82/155] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:24 2022] GET /identity/v3/users/f4ce8d7e0ba24e4ebab4b74a1f28995a => generated 289 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:24.356337 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92ea8d-0d23-4592-952f-fa9ac4a61593 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:24.375054 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 74/156] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:14:24.377070 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:24.467109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1bc57-03c8-4a43-baca-ee7123430579 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:24.517018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92ea8d-0d23-4592-952f-fa9ac4a61593 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:24.574597 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4b4f4-7d16-4b6c-8279-1fc900c548c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:24.575505 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4b4f4-7d16-4b6c-8279-1fc900c548c6 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:24.576666 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4b4f4-7d16-4b6c-8279-1fc900c548c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:24.576948 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4b4f4-7d16-4b6c-8279-1fc900c548c6 None admin] PATH_INFO: `/v3/projects/9dbc862abb4247f69fc1a65f462d65ce` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:24.579601 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4b4f4-7d16-4b6c-8279-1fc900c548c6 None admin] RBAC: Authorizing `identity:get_project(project_id=9dbc862abb4247f69fc1a65f462d65ce)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:24.581256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4b4f4-7d16-4b6c-8279-1fc900c548c6 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:24.583369 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 83/157] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:24 2022] GET /identity/v3/projects/9dbc862abb4247f69fc1a65f462d65ce => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:24.586116 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffbfc6-adb9-48ad-83e1-92fda9015f01 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:24.594497 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffbfc6-adb9-48ad-83e1-92fda9015f01 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:24.596126 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffbfc6-adb9-48ad-83e1-92fda9015f01 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:24.597734 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 84/158] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:24 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:24.598886 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ffbfc6-adb9-48ad-83e1-92fda9015f01 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:24.609307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:24.706547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1bc57-03c8-4a43-baca-ee7123430579 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:24.790769 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:24.794054 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d455da8-7970-4513-a0c5-9fb9b598cce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:24.795091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d455da8-7970-4513-a0c5-9fb9b598cce7 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:24.795369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d455da8-7970-4513-a0c5-9fb9b598cce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:24.795640 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d455da8-7970-4513-a0c5-9fb9b598cce7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:24.817757 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d455da8-7970-4513-a0c5-9fb9b598cce7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:24.832928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-3d455da8-7970-4513-a0c5-9fb9b598cce7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:14:24.832928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:24.832928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:24.832928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:24.832928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:24.832928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:24.832928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:24.832928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:24.832928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:24.832928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:24.832928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:24.832928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:24.832928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:24.832928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:24.832928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:24.832928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:24.833828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:24.833828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:24.833828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:24.833828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:24.833828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:24.833828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:24.833828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:24.833828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:24.833828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:24.833828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:24.833828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:24.833828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:14:24.833828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:24.833828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:24.833828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:24.833828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:24.834673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:24.834673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:24.834673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:24.834673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:24.834673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:24.834673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:24.834673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:24.834673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:24.834673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:24.834673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:24.834673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:14:24.834673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:14:24.836016 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 75/159] 10.222.0.24 () {56 vars in 1114 bytes} [Fri May 27 03:14:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:14:24.836889 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d687f2-b5f6-4038-acb7-dc345f26e132 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:24.837868 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d687f2-b5f6-4038-acb7-dc345f26e132 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:24.840106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d687f2-b5f6-4038-acb7-dc345f26e132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:24.843006 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ffbfc6-adb9-48ad-83e1-92fda9015f01 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:24.884896 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-c7d687f2-b5f6-4038-acb7-dc345f26e132 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:24.885998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c7d687f2-b5f6-4038-acb7-dc345f26e132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:24.914413 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375a9a9-9d70-42c4-a802-7abbfe813eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:24.915429 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375a9a9-9d70-42c4-a802-7abbfe813eee None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:24.915708 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375a9a9-9d70-42c4-a802-7abbfe813eee None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:24.916086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375a9a9-9d70-42c4-a802-7abbfe813eee None admin] PATH_INFO: `/v3/projects/9dbc862abb4247f69fc1a65f462d65ce/users/f4ce8d7e0ba24e4ebab4b74a1f28995a/roles/aa0d3c6b36a24a718f88d0f4309d26d9` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:24.922903 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375a9a9-9d70-42c4-a802-7abbfe813eee None admin] RBAC: Authorizing `identity:create_grant(project_id=9dbc862abb4247f69fc1a65f462d65ce, user_id=f4ce8d7e0ba24e4ebab4b74a1f28995a, role_id=aa0d3c6b36a24a718f88d0f4309d26d9)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:24.924627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375a9a9-9d70-42c4-a802-7abbfe813eee None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:25.010346 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 76/160] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:25.015359 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d687f2-b5f6-4038-acb7-dc345f26e132 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:25.041479 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 85/161] 10.222.0.24 () {58 vars in 1439 bytes} [Fri May 27 03:14:24 2022] PUT /identity/v3/projects/9dbc862abb4247f69fc1a65f462d65ce/users/f4ce8d7e0ba24e4ebab4b74a1f28995a/roles/aa0d3c6b36a24a718f88d0f4309d26d9 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:14:25.050278 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375a9a9-9d70-42c4-a802-7abbfe813eee None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:25.092565 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:25.233019 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d687f2-b5f6-4038-acb7-dc345f26e132 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:25.273193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375a9a9-9d70-42c4-a802-7abbfe813eee None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:25.334863 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479572d-91d9-49df-a3c6-f5c0bde0122d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:25.334863 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479572d-91d9-49df-a3c6-f5c0bde0122d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:25.334863 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479572d-91d9-49df-a3c6-f5c0bde0122d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:25.334863 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479572d-91d9-49df-a3c6-f5c0bde0122d None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:25.335286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0479572d-91d9-49df-a3c6-f5c0bde0122d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:25.335286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0479572d-91d9-49df-a3c6-f5c0bde0122d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:25.336428 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0479572d-91d9-49df-a3c6-f5c0bde0122d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:25.350795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 77/162] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:14:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:25.352232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe247dd7-ea2a-4054-a6b9-875d0fe4b299 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:25.352527 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe247dd7-ea2a-4054-a6b9-875d0fe4b299 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:25.352802 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe247dd7-ea2a-4054-a6b9-875d0fe4b299 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:25.354290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 78/163] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:25.356928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe247dd7-ea2a-4054-a6b9-875d0fe4b299 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:25.373035 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15b59e-d33d-469e-8a3d-46fc8f23c9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:25.373839 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15b59e-d33d-469e-8a3d-46fc8f23c9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:25.374118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15b59e-d33d-469e-8a3d-46fc8f23c9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:25.374385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15b59e-d33d-469e-8a3d-46fc8f23c9e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:25.378671 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:25.395490 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b15b59e-d33d-469e-8a3d-46fc8f23c9e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:25.404310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-9b15b59e-d33d-469e-8a3d-46fc8f23c9e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:14:25.404310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:25.404310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:25.404310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:25.404310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:25.404310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:25.404310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:25.404310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:25.404310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:25.404310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:25.404310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:25.404310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:25.404310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:25.404310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:25.404310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:25.404310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:25.405166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:25.405166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:25.405166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:25.405166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:25.405166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:25.405166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:25.405166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:25.405166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:25.405166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:25.405166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:25.405166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:25.405166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:14:25.405166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:25.405166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:25.405166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:25.405166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:25.406007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:25.406007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:25.406007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:25.406007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:25.406007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:25.406007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:25.406007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:25.406007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:25.406007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:25.406007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:25.406007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:14:25.406007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:14:25.406007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 86/164] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:14:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:25.406958 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e769acd-134d-4fe8-bf59-ca50f4610d0b None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:25.407997 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e769acd-134d-4fe8-bf59-ca50f4610d0b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:25.407997 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e769acd-134d-4fe8-bf59-ca50f4610d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:25.428220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-4e769acd-134d-4fe8-bf59-ca50f4610d0b None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:25.429429 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4e769acd-134d-4fe8-bf59-ca50f4610d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:25.530979 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe247dd7-ea2a-4054-a6b9-875d0fe4b299 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:25.550707 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:25.585190 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e7ae9-e4b6-4801-a618-e247e6931cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:25.586086 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e7ae9-e4b6-4801-a618-e247e6931cda None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:25.586366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e7ae9-e4b6-4801-a618-e247e6931cda None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:25.586639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e7ae9-e4b6-4801-a618-e247e6931cda None admin] PATH_INFO: `/v3/users/3e25056b9cac4b5cb4ff28c737a1cbdd` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:25.587865 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e7ae9-e4b6-4801-a618-e247e6931cda None admin] RBAC: Authorizing `identity:get_user(user_id=3e25056b9cac4b5cb4ff28c737a1cbdd)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:25.589324 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e7ae9-e4b6-4801-a618-e247e6931cda None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:25.591044 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 79/165] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:25 2022] GET /identity/v3/users/3e25056b9cac4b5cb4ff28c737a1cbdd => generated 275 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:25.591883 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e7ae9-e4b6-4801-a618-e247e6931cda None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:25.612188 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 87/166] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:25.613384 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e769acd-134d-4fe8-bf59-ca50f4610d0b None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:25.733479 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e7ae9-e4b6-4801-a618-e247e6931cda None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:25.790959 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796487af-dc24-495d-a383-0b2e471562b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:25.791269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796487af-dc24-495d-a383-0b2e471562b1 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:25.791523 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796487af-dc24-495d-a383-0b2e471562b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:25.791794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796487af-dc24-495d-a383-0b2e471562b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:25.792552 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796487af-dc24-495d-a383-0b2e471562b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:25.792859 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796487af-dc24-495d-a383-0b2e471562b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:25.794242 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796487af-dc24-495d-a383-0b2e471562b1 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:25.812108 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 80/167] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:25.818860 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796487af-dc24-495d-a383-0b2e471562b1 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:25.844668 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e769acd-134d-4fe8-bf59-ca50f4610d0b None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:25.898884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:25.899069 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:25.912950 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b661551-de37-4ce6-acd5-a8b70aae2937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:25.913982 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b661551-de37-4ce6-acd5-a8b70aae2937 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:25.914476 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b661551-de37-4ce6-acd5-a8b70aae2937 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:25.914979 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b661551-de37-4ce6-acd5-a8b70aae2937 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:26.005237 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796487af-dc24-495d-a383-0b2e471562b1 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:26.025359 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b661551-de37-4ce6-acd5-a8b70aae2937 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:26.027207 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b661551-de37-4ce6-acd5-a8b70aae2937 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:26.075654 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c511a6-eebb-43bb-ad1c-4c273d81aba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:26.076659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c511a6-eebb-43bb-ad1c-4c273d81aba6 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:26.076946 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c511a6-eebb-43bb-ad1c-4c273d81aba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:26.077224 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c511a6-eebb-43bb-ad1c-4c273d81aba6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:26.111832 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c511a6-eebb-43bb-ad1c-4c273d81aba6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:26.124316 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-a6c511a6-eebb-43bb-ad1c-4c273d81aba6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:26.124316 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:26.124316 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:26.124316 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:26.124316 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:26.124316 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:26.124316 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:26.124316 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:26.124316 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:26.124316 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:26.124316 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:26.124316 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:26.124316 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:26.124316 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:26.124316 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:26.124316 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:26.125300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:26.125300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:26.125300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:26.125300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:26.125300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:26.125300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:26.125300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:26.125300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:26.125300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:26.125300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:26.125300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:26.125300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:14:26.125300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:26.125300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:26.125300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:26.125300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:26.126082 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:26.126082 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:26.126082 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:26.126082 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:26.126082 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:26.126082 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:26.126082 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:26.126082 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:26.126082 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:26.126082 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:26.126082 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:26.126082 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:14:26.126082 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 81/168] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:26.127234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c511a6-eebb-43bb-ad1c-4c273d81aba6 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:26.161840 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 88/169] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:14:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:14:26.163837 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b661551-de37-4ce6-acd5-a8b70aae2937 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:26.192318 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:26.313412 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c511a6-eebb-43bb-ad1c-4c273d81aba6 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:26.316592 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:26.354570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070cb7d6-7c27-4fee-a506-b5174de1e428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:26.356290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070cb7d6-7c27-4fee-a506-b5174de1e428 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:26.357451 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070cb7d6-7c27-4fee-a506-b5174de1e428 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:26.358036 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070cb7d6-7c27-4fee-a506-b5174de1e428 None admin] PATH_INFO: `/v3/users/93ab1998e69940199adfdba4433a4d08` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:26.360110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070cb7d6-7c27-4fee-a506-b5174de1e428 None admin] RBAC: Authorizing `identity:get_user(user_id=93ab1998e69940199adfdba4433a4d08)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:26.361850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070cb7d6-7c27-4fee-a506-b5174de1e428 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:26.364069 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 82/170] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:26 2022] GET /identity/v3/users/93ab1998e69940199adfdba4433a4d08 => generated 247 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:26.365052 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070cb7d6-7c27-4fee-a506-b5174de1e428 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:26.365813 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b661551-de37-4ce6-acd5-a8b70aae2937 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:26.431558 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ed1d8-9bf4-4e74-af79-0e4399404d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:26.432356 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ed1d8-9bf4-4e74-af79-0e4399404d7c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:26.432643 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ed1d8-9bf4-4e74-af79-0e4399404d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:26.432923 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ed1d8-9bf4-4e74-af79-0e4399404d7c None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:26.433788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ed1d8-9bf4-4e74-af79-0e4399404d7c None admin] RBAC: Adding query filter params (name=member) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:26.434103 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ed1d8-9bf4-4e74-af79-0e4399404d7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:26.436086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ed1d8-9bf4-4e74-af79-0e4399404d7c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:26.456335 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 89/171] 10.222.0.24 () {56 vars in 1101 bytes} [Fri May 27 03:14:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:26.457199 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ed1d8-9bf4-4e74-af79-0e4399404d7c None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:26.498775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070cb7d6-7c27-4fee-a506-b5174de1e428 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:26.556599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a9b03-9349-4928-bb38-972fa55a1ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:26.557493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a9b03-9349-4928-bb38-972fa55a1ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:26.557773 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a9b03-9349-4928-bb38-972fa55a1ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:26.558046 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a9b03-9349-4928-bb38-972fa55a1ad6 None admin] PATH_INFO: `/v3/projects/9dbc862abb4247f69fc1a65f462d65ce` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:26.559291 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a9b03-9349-4928-bb38-972fa55a1ad6 None admin] RBAC: Authorizing `identity:get_project(project_id=9dbc862abb4247f69fc1a65f462d65ce)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:26.561341 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381a9b03-9349-4928-bb38-972fa55a1ad6 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:26.563084 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 83/172] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:26 2022] GET /identity/v3/projects/9dbc862abb4247f69fc1a65f462d65ce => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:26.563862 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a9b03-9349-4928-bb38-972fa55a1ad6 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:26.631354 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:26.669805 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:26.700023 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ed1d8-9bf4-4e74-af79-0e4399404d7c None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:26.736613 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a9b03-9349-4928-bb38-972fa55a1ad6 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:26.759885 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866b84d-16f7-47a1-90e3-991d609253a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:26.760538 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866b84d-16f7-47a1-90e3-991d609253a3 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:26.760817 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866b84d-16f7-47a1-90e3-991d609253a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:26.761090 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866b84d-16f7-47a1-90e3-991d609253a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:26.808880 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b18bd6-00c5-4a55-adbb-ee445f579d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:26.809764 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b18bd6-00c5-4a55-adbb-ee445f579d14 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:26.810582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b18bd6-00c5-4a55-adbb-ee445f579d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:26.810872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b18bd6-00c5-4a55-adbb-ee445f579d14 None admin] PATH_INFO: `/v3/users/3e25056b9cac4b5cb4ff28c737a1cbdd` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:26.812734 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b18bd6-00c5-4a55-adbb-ee445f579d14 None admin] RBAC: Authorizing `identity:get_user(user_id=3e25056b9cac4b5cb4ff28c737a1cbdd)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:26.814192 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b18bd6-00c5-4a55-adbb-ee445f579d14 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:26.821132 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 84/173] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:26 2022] GET /identity/v3/users/3e25056b9cac4b5cb4ff28c737a1cbdd => generated 275 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:26.821960 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b18bd6-00c5-4a55-adbb-ee445f579d14 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:26.890683 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3866b84d-16f7-47a1-90e3-991d609253a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:26.892308 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3866b84d-16f7-47a1-90e3-991d609253a3 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:26.992565 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b18bd6-00c5-4a55-adbb-ee445f579d14 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:27.007418 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:27.026210 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:27.052695 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7daad7d-50ce-4e42-92d1-ac8a26cd5caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:27.053369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7daad7d-50ce-4e42-92d1-ac8a26cd5caa None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:27.053656 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7daad7d-50ce-4e42-92d1-ac8a26cd5caa None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:27.053935 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7daad7d-50ce-4e42-92d1-ac8a26cd5caa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:27.054489 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 90/174] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:14:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:14:27.054899 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7daad7d-50ce-4e42-92d1-ac8a26cd5caa None admin] RBAC: Adding query filter params (role.id=d7133ce917f0431883775551ab086d5d, scope.project.id=9dbc862abb4247f69fc1a65f462d65ce, user.id=3e25056b9cac4b5cb4ff28c737a1cbdd) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:27.055224 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7daad7d-50ce-4e42-92d1-ac8a26cd5caa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:27.055950 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad96a51-68bf-4c1a-a436-7ba2c6135da9 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:27.061627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad96a51-68bf-4c1a-a436-7ba2c6135da9 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:27.061627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad96a51-68bf-4c1a-a436-7ba2c6135da9 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:27.061800 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7daad7d-50ce-4e42-92d1-ac8a26cd5caa None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:27.061874 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 91/175] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:27.061874 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad96a51-68bf-4c1a-a436-7ba2c6135da9 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:27.067675 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 85/176] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:14:26 2022] GET /identity/v3/role_assignments?user.id=3e25056b9cac4b5cb4ff28c737a1cbdd&scope.project.id=9dbc862abb4247f69fc1a65f462d65ce&role.id=d7133ce917f0431883775551ab086d5d => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:27.068517 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7daad7d-50ce-4e42-92d1-ac8a26cd5caa None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:27.224828 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad96a51-68bf-4c1a-a436-7ba2c6135da9 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:27.267430 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa413b81-6a48-452f-beb0-e4f399318300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:27.268107 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa413b81-6a48-452f-beb0-e4f399318300 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:27.268394 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa413b81-6a48-452f-beb0-e4f399318300 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:27.268674 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa413b81-6a48-452f-beb0-e4f399318300 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:27.303963 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7daad7d-50ce-4e42-92d1-ac8a26cd5caa None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:27.344950 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:27.347599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd3fd40-81f9-433c-b50c-8b99f16d4935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:27.348769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3fd40-81f9-433c-b50c-8b99f16d4935 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:27.349054 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3fd40-81f9-433c-b50c-8b99f16d4935 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:27.349468 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3fd40-81f9-433c-b50c-8b99f16d4935 None admin] PATH_INFO: `/v3/projects/9dbc862abb4247f69fc1a65f462d65ce` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:27.350795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd3fd40-81f9-433c-b50c-8b99f16d4935 None admin] RBAC: Authorizing `identity:get_project(project_id=9dbc862abb4247f69fc1a65f462d65ce)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:27.352280 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd3fd40-81f9-433c-b50c-8b99f16d4935 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:27.354001 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 86/177] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:27 2022] GET /identity/v3/projects/9dbc862abb4247f69fc1a65f462d65ce => generated 298 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:27.355178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98762ea6-c1dc-43b6-bc5e-89b7b74d4ad5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:27.355460 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98762ea6-c1dc-43b6-bc5e-89b7b74d4ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:27.355734 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98762ea6-c1dc-43b6-bc5e-89b7b74d4ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:27.358818 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa413b81-6a48-452f-beb0-e4f399318300 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:27.358818 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa413b81-6a48-452f-beb0-e4f399318300 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:27.379235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-98762ea6-c1dc-43b6-bc5e-89b7b74d4ad5 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:27.380425 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-98762ea6-c1dc-43b6-bc5e-89b7b74d4ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:27.421085 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:27.456368 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 87/178] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:27.458770 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8dcce-78d8-4f7f-b8b7-19f5c2081279 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:27.459134 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8dcce-78d8-4f7f-b8b7-19f5c2081279 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:27.459420 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8dcce-78d8-4f7f-b8b7-19f5c2081279 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:27.460861 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 88/179] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:27.471426 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b100e-e402-482a-aa45-aa57ea20a776 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:27.471426 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b100e-e402-482a-aa45-aa57ea20a776 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:27.471426 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b100e-e402-482a-aa45-aa57ea20a776 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:27.471426 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 89/180] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:27.471426 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b100e-e402-482a-aa45-aa57ea20a776 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:27.482816 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 92/181] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:14:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:14:27.484716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ef671-30f4-43f8-a108-8f2edcf2549d None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:27.485179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ef671-30f4-43f8-a108-8f2edcf2549d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:27.485831 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ef671-30f4-43f8-a108-8f2edcf2549d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:27.503099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-c85ef671-30f4-43f8-a108-8f2edcf2549d None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:27.504314 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c85ef671-30f4-43f8-a108-8f2edcf2549d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:27.580944 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 93/182] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:27.582305 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680cef7-6c8c-4b57-a096-8ea5a84e96e5 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:27.582594 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680cef7-6c8c-4b57-a096-8ea5a84e96e5 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:27.582885 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680cef7-6c8c-4b57-a096-8ea5a84e96e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:27.608416 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-9680cef7-6c8c-4b57-a096-8ea5a84e96e5 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:27.609742 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-9680cef7-6c8c-4b57-a096-8ea5a84e96e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:27.611366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b100e-e402-482a-aa45-aa57ea20a776 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:27.625504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:27.645401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdfef1a-e735-4c52-ba69-0f2374ba838e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:27.646172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdfef1a-e735-4c52-ba69-0f2374ba838e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:27.646569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdfef1a-e735-4c52-ba69-0f2374ba838e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:27.646901 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdfef1a-e735-4c52-ba69-0f2374ba838e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:27.647733 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdfef1a-e735-4c52-ba69-0f2374ba838e None admin] RBAC: Adding query filter params (role.id=e5a908bd3ed5445b82a6053246533b88, scope.project.id=9dbc862abb4247f69fc1a65f462d65ce, user.id=93ab1998e69940199adfdba4433a4d08) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:27.648040 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdfef1a-e735-4c52-ba69-0f2374ba838e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:27.649504 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdfef1a-e735-4c52-ba69-0f2374ba838e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:27.666882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 90/183] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:14:27 2022] GET /identity/v3/role_assignments?user.id=93ab1998e69940199adfdba4433a4d08&scope.project.id=9dbc862abb4247f69fc1a65f462d65ce&role.id=e5a908bd3ed5445b82a6053246533b88 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:27.666882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdfef1a-e735-4c52-ba69-0f2374ba838e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:27.703431 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 94/184] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:27.704436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680cef7-6c8c-4b57-a096-8ea5a84e96e5 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:27.705947 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:27.792846 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdfef1a-e735-4c52-ba69-0f2374ba838e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:27.831323 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b6ccf-e73f-409d-8c7a-f99e6262ff0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:27.832224 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b6ccf-e73f-409d-8c7a-f99e6262ff0e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:27.832697 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b6ccf-e73f-409d-8c7a-f99e6262ff0e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:27.833142 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840b6ccf-e73f-409d-8c7a-f99e6262ff0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:27.844608 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680cef7-6c8c-4b57-a096-8ea5a84e96e5 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:27.845789 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840b6ccf-e73f-409d-8c7a-f99e6262ff0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:27.853749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-840b6ccf-e73f-409d-8c7a-f99e6262ff0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:14:27.853749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:27.853749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:27.853749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:27.853749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:27.853749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:27.853749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:27.853749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:27.853749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:27.853749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:27.853749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:27.853749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:27.853749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:27.853749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:27.853749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:27.853749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:27.854680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:27.854680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:27.854680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:27.854680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:27.854680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:27.854680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:27.854680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:27.854680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:27.854680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:27.854680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:27.854680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:27.854680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:14:27.854680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:27.854680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:27.854680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:27.854680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:27.855491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:27.855491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:27.855491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:27.855491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:27.855491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:27.855491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:27.855491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:27.855491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:27.855491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:27.855491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:27.855491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:14:27.855491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:14:27.858609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 91/185] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:14:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:27.864168 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b6ccf-e73f-409d-8c7a-f99e6262ff0e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:27.885632 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42bd5a3-c941-4910-8263-83deafed0826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:27.887003 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42bd5a3-c941-4910-8263-83deafed0826 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:27.887480 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42bd5a3-c941-4910-8263-83deafed0826 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:27.888616 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42bd5a3-c941-4910-8263-83deafed0826 None admin] PATH_INFO: `/v3/projects/ba26937020c4422ca6807b1bf79820b5` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:27.890202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42bd5a3-c941-4910-8263-83deafed0826 None admin] RBAC: Authorizing `identity:get_project(project_id=ba26937020c4422ca6807b1bf79820b5)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:27.892353 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42bd5a3-c941-4910-8263-83deafed0826 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:27.900117 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 95/186] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:27 2022] GET /identity/v3/projects/ba26937020c4422ca6807b1bf79820b5 => generated 312 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:27.901577 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42bd5a3-c941-4910-8263-83deafed0826 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:27.923687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:27.976857 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:27.984423 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840b6ccf-e73f-409d-8c7a-f99e6262ff0e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:28.030302 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5d543-4521-43e7-922d-c52ca0bffe97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:28.031182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5d543-4521-43e7-922d-c52ca0bffe97 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:28.031470 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5d543-4521-43e7-922d-c52ca0bffe97 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:28.031747 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5d543-4521-43e7-922d-c52ca0bffe97 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:28.032873 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42bd5a3-c941-4910-8263-83deafed0826 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:28.043016 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e5d543-4521-43e7-922d-c52ca0bffe97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:28.051387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-d7e5d543-4521-43e7-922d-c52ca0bffe97 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:14:28.051387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:28.051387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:28.051387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:28.051387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:28.051387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:28.051387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:28.051387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:28.051387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:28.051387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:28.051387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:28.051387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:28.051387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:28.051387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:28.051387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:28.051387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:28.052327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:28.052327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:28.052327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:28.052327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:28.052327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:28.052327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:28.052327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:28.052327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:28.052327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:28.052327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:28.052327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:28.052327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:14:28.052327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:28.052327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:28.052327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:28.052327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:28.053085 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:28.053085 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:28.053085 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:28.053085 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:28.053085 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:28.053085 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:28.053085 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:28.053085 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:28.053085 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:28.053085 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:28.053085 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:14:28.053085 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:14:28.057269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 92/187] 10.222.0.24 () {56 vars in 1114 bytes} [Fri May 27 03:14:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:28.058053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5d543-4521-43e7-922d-c52ca0bffe97 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:28.072648 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671c9e1-8dac-4ce3-bed5-87b972a58f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:28.073841 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671c9e1-8dac-4ce3-bed5-87b972a58f53 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:28.074329 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671c9e1-8dac-4ce3-bed5-87b972a58f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:28.074766 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671c9e1-8dac-4ce3-bed5-87b972a58f53 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:28.077001 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8671c9e1-8dac-4ce3-bed5-87b972a58f53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:28.078745 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8671c9e1-8dac-4ce3-bed5-87b972a58f53 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:28.080713 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 96/188] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:14:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:28.081672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671c9e1-8dac-4ce3-bed5-87b972a58f53 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:28.177225 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:28.183518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e5d543-4521-43e7-922d-c52ca0bffe97 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:28.192477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671c9e1-8dac-4ce3-bed5-87b972a58f53 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:28.219252 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcaee20-df49-488d-8514-bcdb4ebf1753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:28.220147 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcaee20-df49-488d-8514-bcdb4ebf1753 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:28.220571 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcaee20-df49-488d-8514-bcdb4ebf1753 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:28.221012 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcaee20-df49-488d-8514-bcdb4ebf1753 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:28.221902 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcaee20-df49-488d-8514-bcdb4ebf1753 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:28.222364 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcaee20-df49-488d-8514-bcdb4ebf1753 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:28.223775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcaee20-df49-488d-8514-bcdb4ebf1753 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:28.227789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f180aa-06fa-4614-8388-8317248bb33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:28.228441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f180aa-06fa-4614-8388-8317248bb33c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:28.228759 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f180aa-06fa-4614-8388-8317248bb33c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:28.228997 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f180aa-06fa-4614-8388-8317248bb33c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:28.229824 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f180aa-06fa-4614-8388-8317248bb33c None admin] RBAC: Adding query filter params (role.id=343337c6f25943a9ad412047b70ddefa, scope.project.id=ba26937020c4422ca6807b1bf79820b5, user.id=3e25056b9cac4b5cb4ff28c737a1cbdd) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:28.230139 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f180aa-06fa-4614-8388-8317248bb33c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:28.231585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f180aa-06fa-4614-8388-8317248bb33c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:28.236993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 93/189] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:28.236993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcaee20-df49-488d-8514-bcdb4ebf1753 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:28.241869 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 97/190] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:14:28 2022] GET /identity/v3/role_assignments?user.id=3e25056b9cac4b5cb4ff28c737a1cbdd&scope.project.id=ba26937020c4422ca6807b1bf79820b5&role.id=343337c6f25943a9ad412047b70ddefa => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:28.251110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f180aa-06fa-4614-8388-8317248bb33c None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:28.268645 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:28.369342 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcaee20-df49-488d-8514-bcdb4ebf1753 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:28.370513 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f180aa-06fa-4614-8388-8317248bb33c None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:28.408122 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a1734-90d6-4266-8eeb-418fdd61c523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:28.408812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a1734-90d6-4266-8eeb-418fdd61c523 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:28.409094 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a1734-90d6-4266-8eeb-418fdd61c523 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:28.409363 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a1734-90d6-4266-8eeb-418fdd61c523 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:28.410334 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a1734-90d6-4266-8eeb-418fdd61c523 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:28.410599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a1734-90d6-4266-8eeb-418fdd61c523 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:28.412064 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a1734-90d6-4266-8eeb-418fdd61c523 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:28.413145 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378df72b-abd5-41cd-bd16-3257dae9b68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:28.414142 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378df72b-abd5-41cd-bd16-3257dae9b68c None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:28.414909 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378df72b-abd5-41cd-bd16-3257dae9b68c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:28.415206 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378df72b-abd5-41cd-bd16-3257dae9b68c None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:28.415941 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378df72b-abd5-41cd-bd16-3257dae9b68c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:28.417424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378df72b-abd5-41cd-bd16-3257dae9b68c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:28.430378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 94/191] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:14:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:28.430378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a1734-90d6-4266-8eeb-418fdd61c523 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:28.514907 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:28.534324 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 98/192] 10.222.0.24 () {60 vars in 1132 bytes} [Fri May 27 03:14:28 2022] POST /identity/v3/users => generated 297 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:14:28.535630 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a1734-90d6-4266-8eeb-418fdd61c523 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:28.535736 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56ad85-352d-4e0f-9c68-4c316ca95b60 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:28.535987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56ad85-352d-4e0f-9c68-4c316ca95b60 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:28.536304 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56ad85-352d-4e0f-9c68-4c316ca95b60 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:28.537693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 99/193] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:28.538283 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e56ad85-352d-4e0f-9c68-4c316ca95b60 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:28.543834 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:28.577482 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6dd420-525f-4838-a9a9-0a484d01dc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:28.578377 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6dd420-525f-4838-a9a9-0a484d01dc3e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:28.578654 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6dd420-525f-4838-a9a9-0a484d01dc3e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:28.578957 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6dd420-525f-4838-a9a9-0a484d01dc3e None admin] PATH_INFO: `/v3/users/f27304a61be64f9e8cab970a5298641f` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:28.595084 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6dd420-525f-4838-a9a9-0a484d01dc3e None admin] RBAC: Authorizing `identity:get_user(user_id=f27304a61be64f9e8cab970a5298641f)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:28.596583 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6dd420-525f-4838-a9a9-0a484d01dc3e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:28.598717 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 95/194] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:28 2022] GET /identity/v3/users/f27304a61be64f9e8cab970a5298641f => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:28.599709 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637ca81-19d4-4d74-9862-340c8294391c None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:28.599993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637ca81-19d4-4d74-9862-340c8294391c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:28.600263 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c637ca81-19d4-4d74-9862-340c8294391c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:28.618762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-c637ca81-19d4-4d74-9862-340c8294391c None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:28.621609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c637ca81-19d4-4d74-9862-340c8294391c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:28.660101 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e56ad85-352d-4e0f-9c68-4c316ca95b60 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:28.702866 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 96/195] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:28.702866 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637ca81-19d4-4d74-9862-340c8294391c None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:28.704384 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c911c-2171-4774-9823-5368c8ed5fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:28.705362 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c911c-2171-4774-9823-5368c8ed5fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:28.705727 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c911c-2171-4774-9823-5368c8ed5fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:28.706052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c911c-2171-4774-9823-5368c8ed5fa4 None admin] PATH_INFO: `/v3/users/f27304a61be64f9e8cab970a5298641f` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:28.707452 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72c911c-2171-4774-9823-5368c8ed5fa4 None admin] RBAC: Authorizing `identity:get_user(user_id=f27304a61be64f9e8cab970a5298641f)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:28.709019 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72c911c-2171-4774-9823-5368c8ed5fa4 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:28.710905 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 100/196] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:28 2022] GET /identity/v3/users/f27304a61be64f9e8cab970a5298641f => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:28.716662 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c911c-2171-4774-9823-5368c8ed5fa4 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:28.776476 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:28.797530 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:28.814515 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c637ca81-19d4-4d74-9862-340c8294391c None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:28.841990 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c911c-2171-4774-9823-5368c8ed5fa4 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:28.861144 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5c7cd-7e07-4d70-9e30-3f0a08bf9c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:28.862320 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5c7cd-7e07-4d70-9e30-3f0a08bf9c8c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:28.862608 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5c7cd-7e07-4d70-9e30-3f0a08bf9c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:28.863207 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc5c7cd-7e07-4d70-9e30-3f0a08bf9c8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:28.885249 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9248f6-dc19-4b02-8f91-8ded2ca06757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:28.886179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9248f6-dc19-4b02-8f91-8ded2ca06757 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:28.886627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9248f6-dc19-4b02-8f91-8ded2ca06757 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:28.886932 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9248f6-dc19-4b02-8f91-8ded2ca06757 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:28.937356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc5c7cd-7e07-4d70-9e30-3f0a08bf9c8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:28.939173 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc5c7cd-7e07-4d70-9e30-3f0a08bf9c8c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:28.974927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9248f6-dc19-4b02-8f91-8ded2ca06757 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:28.977082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9248f6-dc19-4b02-8f91-8ded2ca06757 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:29.041614 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:29.091512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 97/197] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:14:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:14:29.091601 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:29.095340 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5c7cd-7e07-4d70-9e30-3f0a08bf9c8c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:29.135569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 101/198] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:14:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:14:29.136134 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9248f6-dc19-4b02-8f91-8ded2ca06757 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:29.216020 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc5c7cd-7e07-4d70-9e30-3f0a08bf9c8c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:29.254043 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ebe3bb-e1a8-4311-aea7-acc57344c4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:29.255122 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ebe3bb-e1a8-4311-aea7-acc57344c4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:29.255406 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ebe3bb-e1a8-4311-aea7-acc57344c4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:29.255683 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ebe3bb-e1a8-4311-aea7-acc57344c4a2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:29.259043 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9248f6-dc19-4b02-8f91-8ded2ca06757 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:29.266889 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ebe3bb-e1a8-4311-aea7-acc57344c4a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:29.274582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-76ebe3bb-e1a8-4311-aea7-acc57344c4a2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:14:29.274582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:29.274582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:29.274582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:29.274582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:29.274582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:29.274582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:29.274582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:29.274582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:29.274582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:29.274582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:29.274582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:29.274582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:29.274582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:29.274582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:29.274582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:29.275493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:29.275493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:29.275493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:29.275493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:29.275493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:29.275493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:29.275493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:29.275493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:29.275493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:29.275493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:29.275493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:29.275493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:14:29.275493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:29.275493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:29.275493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:29.275493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:29.276262 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:29.276262 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:29.276262 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:29.276262 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:29.276262 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:29.276262 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:29.276262 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:29.276262 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:29.276262 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:29.276262 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:29.276262 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:14:29.276262 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:14:29.276262 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 98/199] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:29.277082 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ebe3bb-e1a8-4311-aea7-acc57344c4a2 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:29.305614 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace193a3-85ea-445e-8934-99189128f5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:29.307239 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace193a3-85ea-445e-8934-99189128f5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:29.307929 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace193a3-85ea-445e-8934-99189128f5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:29.308345 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace193a3-85ea-445e-8934-99189128f5b5 None admin] PATH_INFO: `/v3/projects/d95eae40782d41759de176dea214eb16` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:29.310879 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace193a3-85ea-445e-8934-99189128f5b5 None admin] RBAC: Authorizing `identity:get_project(project_id=d95eae40782d41759de176dea214eb16)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:29.312639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:29.314345 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace193a3-85ea-445e-8934-99189128f5b5 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:29.316905 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 102/200] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:29 2022] GET /identity/v3/projects/d95eae40782d41759de176dea214eb16 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:29.318118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace193a3-85ea-445e-8934-99189128f5b5 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:29.371634 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:29.403081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ebe3bb-e1a8-4311-aea7-acc57344c4a2 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:29.440017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed0e6d-cdd5-49f6-8885-88b3939ef595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:29.441064 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed0e6d-cdd5-49f6-8885-88b3939ef595 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:29.441254 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed0e6d-cdd5-49f6-8885-88b3939ef595 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:29.441649 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed0e6d-cdd5-49f6-8885-88b3939ef595 None admin] PATH_INFO: `/v3/projects/d95eae40782d41759de176dea214eb16` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:29.442941 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed0e6d-cdd5-49f6-8885-88b3939ef595 None admin] RBAC: Authorizing `identity:get_project(project_id=d95eae40782d41759de176dea214eb16)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:29.444484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ed0e6d-cdd5-49f6-8885-88b3939ef595 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:29.446315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 99/201] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:29 2022] GET /identity/v3/projects/d95eae40782d41759de176dea214eb16 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:29.447288 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed0e6d-cdd5-49f6-8885-88b3939ef595 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:29.453714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace193a3-85ea-445e-8934-99189128f5b5 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:29.498508 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bdb87b-fe94-4dfe-9d5a-e14319d57e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:29.499303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdb87b-fe94-4dfe-9d5a-e14319d57e54 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:29.499657 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdb87b-fe94-4dfe-9d5a-e14319d57e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:29.500001 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdb87b-fe94-4dfe-9d5a-e14319d57e54 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:29.500825 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bdb87b-fe94-4dfe-9d5a-e14319d57e54 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:29.501250 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bdb87b-fe94-4dfe-9d5a-e14319d57e54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:29.502750 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bdb87b-fe94-4dfe-9d5a-e14319d57e54 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:29.512245 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 103/202] 10.222.0.24 () {56 vars in 1101 bytes} [Fri May 27 03:14:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:29.513407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bdb87b-fe94-4dfe-9d5a-e14319d57e54 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:29.573153 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:29.581078 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed0e6d-cdd5-49f6-8885-88b3939ef595 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:29.617447 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba549dd9-31eb-4959-b85b-807d48dd1e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:29.618369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba549dd9-31eb-4959-b85b-807d48dd1e2d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:29.618828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba549dd9-31eb-4959-b85b-807d48dd1e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:29.619131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bdb87b-fe94-4dfe-9d5a-e14319d57e54 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:29.619518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba549dd9-31eb-4959-b85b-807d48dd1e2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:29.620525 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba549dd9-31eb-4959-b85b-807d48dd1e2d None admin] RBAC: Adding query filter params (role.id=e5a908bd3ed5445b82a6053246533b88, scope.project.id=d95eae40782d41759de176dea214eb16, user.id=f27304a61be64f9e8cab970a5298641f) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:29.620917 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:29.621417 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba549dd9-31eb-4959-b85b-807d48dd1e2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:29.624261 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba549dd9-31eb-4959-b85b-807d48dd1e2d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:29.630211 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 100/203] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:14:29 2022] GET /identity/v3/role_assignments?user.id=f27304a61be64f9e8cab970a5298641f&scope.project.id=d95eae40782d41759de176dea214eb16&role.id=e5a908bd3ed5445b82a6053246533b88 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:29.631205 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba549dd9-31eb-4959-b85b-807d48dd1e2d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:29.657192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04528bbe-2c12-432e-99db-610d3171d89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:29.657897 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04528bbe-2c12-432e-99db-610d3171d89d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:29.658293 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04528bbe-2c12-432e-99db-610d3171d89d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:29.658669 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04528bbe-2c12-432e-99db-610d3171d89d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:29.659922 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04528bbe-2c12-432e-99db-610d3171d89d None admin] RBAC: Adding query filter params (role.id=d7133ce917f0431883775551ab086d5d, scope.project.id=d95eae40782d41759de176dea214eb16, user.id=f27304a61be64f9e8cab970a5298641f) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:29.660587 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04528bbe-2c12-432e-99db-610d3171d89d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:29.662444 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04528bbe-2c12-432e-99db-610d3171d89d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:29.669614 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 104/204] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:14:29 2022] GET /identity/v3/role_assignments?user.id=f27304a61be64f9e8cab970a5298641f&scope.project.id=d95eae40782d41759de176dea214eb16&role.id=d7133ce917f0431883775551ab086d5d => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:29.788996 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba549dd9-31eb-4959-b85b-807d48dd1e2d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:29.839832 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364e22a-3357-41f6-8e93-c2ce79a684a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:29.841517 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364e22a-3357-41f6-8e93-c2ce79a684a7 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:29.841944 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364e22a-3357-41f6-8e93-c2ce79a684a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:29.842382 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364e22a-3357-41f6-8e93-c2ce79a684a7 None admin] PATH_INFO: `/v3/users/f4ce8d7e0ba24e4ebab4b74a1f28995a` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:29.843821 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7364e22a-3357-41f6-8e93-c2ce79a684a7 None admin] RBAC: Authorizing `identity:get_user(user_id=f4ce8d7e0ba24e4ebab4b74a1f28995a)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:29.845468 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7364e22a-3357-41f6-8e93-c2ce79a684a7 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:29.847723 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 101/205] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:29 2022] GET /identity/v3/users/f4ce8d7e0ba24e4ebab4b74a1f28995a => generated 289 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:29.853136 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04528bbe-2c12-432e-99db-610d3171d89d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:29.978613 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04528bbe-2c12-432e-99db-610d3171d89d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:30.083682 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf626d40-ac13-49ba-91aa-a876aada8125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:30.084358 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf626d40-ac13-49ba-91aa-a876aada8125 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:30.084637 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf626d40-ac13-49ba-91aa-a876aada8125 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:30.084909 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf626d40-ac13-49ba-91aa-a876aada8125 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:30.133260 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:30.176901 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf626d40-ac13-49ba-91aa-a876aada8125 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:30.178430 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf626d40-ac13-49ba-91aa-a876aada8125 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:30.320143 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 105/206] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:14:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:14:30.326148 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364e22a-3357-41f6-8e93-c2ce79a684a7 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:30.379582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:30.479534 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364e22a-3357-41f6-8e93-c2ce79a684a7 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:30.520811 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b54ac7-7d49-4593-9740-da08e1b8ee30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:30.522091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b54ac7-7d49-4593-9740-da08e1b8ee30 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:30.522571 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b54ac7-7d49-4593-9740-da08e1b8ee30 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:30.523103 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b54ac7-7d49-4593-9740-da08e1b8ee30 None admin] PATH_INFO: `/v3/projects/9dbc862abb4247f69fc1a65f462d65ce` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:30.524509 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b54ac7-7d49-4593-9740-da08e1b8ee30 None admin] RBAC: Authorizing `identity:get_project(project_id=9dbc862abb4247f69fc1a65f462d65ce)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:30.526150 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b54ac7-7d49-4593-9740-da08e1b8ee30 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:30.528432 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 102/207] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:30 2022] GET /identity/v3/projects/9dbc862abb4247f69fc1a65f462d65ce => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:30.533998 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf626d40-ac13-49ba-91aa-a876aada8125 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:30.625011 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:30.673558 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf626d40-ac13-49ba-91aa-a876aada8125 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:30.720776 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a750d-c462-4652-af59-c8fcc9cc13d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:30.721910 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a750d-c462-4652-af59-c8fcc9cc13d2 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:30.722436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a750d-c462-4652-af59-c8fcc9cc13d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:30.722924 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a750d-c462-4652-af59-c8fcc9cc13d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:30.723968 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021a750d-c462-4652-af59-c8fcc9cc13d2 None admin] RBAC: Adding query filter params (role.id=aa0d3c6b36a24a718f88d0f4309d26d9, scope.project.id=9dbc862abb4247f69fc1a65f462d65ce, user.id=f4ce8d7e0ba24e4ebab4b74a1f28995a) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:30.724474 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021a750d-c462-4652-af59-c8fcc9cc13d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:30.726158 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021a750d-c462-4652-af59-c8fcc9cc13d2 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:30.739429 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 106/208] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:14:30 2022] GET /identity/v3/role_assignments?user.id=f4ce8d7e0ba24e4ebab4b74a1f28995a&scope.project.id=9dbc862abb4247f69fc1a65f462d65ce&role.id=aa0d3c6b36a24a718f88d0f4309d26d9 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:31.088043 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea77a5-ece7-48d5-b9b0-34e667c0f0c0 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:31.088347 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea77a5-ece7-48d5-b9b0-34e667c0f0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:31.088599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea77a5-ece7-48d5-b9b0-34e667c0f0c0 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:31.090284 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 103/209] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:31.091877 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e7874-8976-49fd-8453-6b876bd9cf1f None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:31.092307 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e7874-8976-49fd-8453-6b876bd9cf1f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:31.092471 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e7874-8976-49fd-8453-6b876bd9cf1f None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:31.093748 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 107/210] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:31.095379 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fefa0ec-5e1d-400a-bb99-dd1dd0b56815 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:31.095661 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fefa0ec-5e1d-400a-bb99-dd1dd0b56815 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:31.095931 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fefa0ec-5e1d-400a-bb99-dd1dd0b56815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:31.102257 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ab98f-ce18-41e9-8495-9e561709a7a8 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:31.102535 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ab98f-ce18-41e9-8495-9e561709a7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:31.102805 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941ab98f-ce18-41e9-8495-9e561709a7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:31.119044 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-4fefa0ec-5e1d-400a-bb99-dd1dd0b56815 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:31.120224 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-4fefa0ec-5e1d-400a-bb99-dd1dd0b56815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:31.129639 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-941ab98f-ce18-41e9-8495-9e561709a7a8 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:31.131221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-941ab98f-ce18-41e9-8495-9e561709a7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:31.208399 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 104/211] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:31.223516 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 108/212] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:31.238969 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fefa0ec-5e1d-400a-bb99-dd1dd0b56815 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:31.256751 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ab98f-ce18-41e9-8495-9e561709a7a8 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:31.271390 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:31.330196 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:31.369706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fefa0ec-5e1d-400a-bb99-dd1dd0b56815 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:31.373021 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941ab98f-ce18-41e9-8495-9e561709a7a8 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:31.413575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1144e9-e508-412e-aa7d-f52846afcc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:31.414544 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1144e9-e508-412e-aa7d-f52846afcc8a None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:31.414981 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1144e9-e508-412e-aa7d-f52846afcc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:31.415361 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1144e9-e508-412e-aa7d-f52846afcc8a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:31.417837 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483bdced-b00d-4963-be50-21e70b633539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:31.419256 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483bdced-b00d-4963-be50-21e70b633539 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:31.419681 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483bdced-b00d-4963-be50-21e70b633539 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:31.420097 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483bdced-b00d-4963-be50-21e70b633539 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:31.421670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483bdced-b00d-4963-be50-21e70b633539 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:31.423602 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483bdced-b00d-4963-be50-21e70b633539 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:31.426204 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 105/213] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:14:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:31.430929 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1144e9-e508-412e-aa7d-f52846afcc8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:31.435358 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-6c1144e9-e508-412e-aa7d-f52846afcc8a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:31.435358 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:31.435358 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:31.435358 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:31.435358 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:31.435358 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:31.435358 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:31.435358 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:31.435358 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:31.435358 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:31.435358 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:31.435358 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:31.435358 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:31.435358 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:31.435358 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:31.435358 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:31.436221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:31.436221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:31.436221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:31.436221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:31.436221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:31.436221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:31.436221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:31.436221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:31.436221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:31.436221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:31.436221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:31.436221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:14:31.436221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:31.436221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:31.436221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:31.436221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:31.436980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:31.436980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:31.436980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:31.436980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:31.436980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:31.436980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:31.436980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:31.436980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:31.436980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:31.436980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:31.436980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:31.436980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:14:31.436980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 109/214] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:14:31.438156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483bdced-b00d-4963-be50-21e70b633539 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:31.442288 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1144e9-e508-412e-aa7d-f52846afcc8a None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:31.556127 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:31.565675 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483bdced-b00d-4963-be50-21e70b633539 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:31.569289 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1144e9-e508-412e-aa7d-f52846afcc8a None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:31.597323 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:31.604696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb1a5e3-7d0c-4097-b83b-6ff67e1384d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:31.605428 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1a5e3-7d0c-4097-b83b-6ff67e1384d1 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:31.605765 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1a5e3-7d0c-4097-b83b-6ff67e1384d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:31.606261 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb1a5e3-7d0c-4097-b83b-6ff67e1384d1 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:31.607304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb1a5e3-7d0c-4097-b83b-6ff67e1384d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:31.608919 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb1a5e3-7d0c-4097-b83b-6ff67e1384d1 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:31.614757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363587b-af0f-4b39-9969-35d7bc734777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:31.615614 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363587b-af0f-4b39-9969-35d7bc734777 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:31.616039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363587b-af0f-4b39-9969-35d7bc734777 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:31.616452 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363587b-af0f-4b39-9969-35d7bc734777 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:31.617636 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363587b-af0f-4b39-9969-35d7bc734777 None admin] RBAC: Adding query filter params (name=member) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:31.618091 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363587b-af0f-4b39-9969-35d7bc734777 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:31.620263 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363587b-af0f-4b39-9969-35d7bc734777 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:31.631916 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 110/215] 10.222.0.24 () {56 vars in 1101 bytes} [Fri May 27 03:14:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:31.637167 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363587b-af0f-4b39-9969-35d7bc734777 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:31.726249 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 106/216] 10.222.0.24 () {60 vars in 1132 bytes} [Fri May 27 03:14:31 2022] POST /identity/v3/users => generated 297 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:14:31.740797 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363587b-af0f-4b39-9969-35d7bc734777 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:31.779851 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326313fb-cb57-410d-a585-b865dbf8d14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:31.780749 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326313fb-cb57-410d-a585-b865dbf8d14c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:31.781029 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326313fb-cb57-410d-a585-b865dbf8d14c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:31.781307 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326313fb-cb57-410d-a585-b865dbf8d14c None admin] PATH_INFO: `/v3/users/f25a34bc71ea41118a7a7546903e977e` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:31.800092 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326313fb-cb57-410d-a585-b865dbf8d14c None admin] RBAC: Authorizing `identity:get_user(user_id=f25a34bc71ea41118a7a7546903e977e)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:31.801563 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326313fb-cb57-410d-a585-b865dbf8d14c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:31.803550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 111/217] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:31 2022] GET /identity/v3/users/f25a34bc71ea41118a7a7546903e977e => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:31.809287 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb1a5e3-7d0c-4097-b83b-6ff67e1384d1 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:31.945504 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb1a5e3-7d0c-4097-b83b-6ff67e1384d1 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:31.976432 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57018a41-8017-4485-9eee-56259bcfaf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:31.977158 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57018a41-8017-4485-9eee-56259bcfaf49 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:31.977436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57018a41-8017-4485-9eee-56259bcfaf49 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:31.977707 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57018a41-8017-4485-9eee-56259bcfaf49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:31.997824 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:32.051427 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57018a41-8017-4485-9eee-56259bcfaf49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:32.052973 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57018a41-8017-4485-9eee-56259bcfaf49 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:32.183531 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 107/218] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:14:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:14:32.196104 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326313fb-cb57-410d-a585-b865dbf8d14c None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:32.255400 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:32.345640 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326313fb-cb57-410d-a585-b865dbf8d14c None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:32.392427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371bbcb9-279c-4df6-9cc4-9b282d56b266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:32.393296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371bbcb9-279c-4df6-9cc4-9b282d56b266 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:32.393573 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371bbcb9-279c-4df6-9cc4-9b282d56b266 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:32.393842 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371bbcb9-279c-4df6-9cc4-9b282d56b266 None admin] PATH_INFO: `/v3/projects/d95eae40782d41759de176dea214eb16` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:32.395072 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371bbcb9-279c-4df6-9cc4-9b282d56b266 None admin] RBAC: Authorizing `identity:get_project(project_id=d95eae40782d41759de176dea214eb16)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:32.395745 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab1360-e603-455a-b389-a9210de2887f None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:32.396510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371bbcb9-279c-4df6-9cc4-9b282d56b266 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:32.396797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab1360-e603-455a-b389-a9210de2887f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:32.397290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab1360-e603-455a-b389-a9210de2887f None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:32.398477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 112/219] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:32 2022] GET /identity/v3/projects/d95eae40782d41759de176dea214eb16 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:32.399240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 108/220] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:32.404899 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047d489-3abe-4736-ad3d-1d07e5bf7041 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:32.405182 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047d489-3abe-4736-ad3d-1d07e5bf7041 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:32.405461 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047d489-3abe-4736-ad3d-1d07e5bf7041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:32.405564 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab1360-e603-455a-b389-a9210de2887f None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:32.435690 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-6047d489-3abe-4736-ad3d-1d07e5bf7041 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:32.437956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6047d489-3abe-4736-ad3d-1d07e5bf7041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:32.496969 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:32.548349 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 113/221] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:14:32.549762 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16f9b9-2062-4236-8804-f4e3873075d1 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:32.550046 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16f9b9-2062-4236-8804-f4e3873075d1 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:32.550318 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da16f9b9-2062-4236-8804-f4e3873075d1 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:32.552680 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 114/222] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:32.558016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c473fd0-b97b-40e0-a8c0-c05d3d260e1d None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:32.558289 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c473fd0-b97b-40e0-a8c0-c05d3d260e1d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:32.558560 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c473fd0-b97b-40e0-a8c0-c05d3d260e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:32.566645 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ab1360-e603-455a-b389-a9210de2887f None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:32.575824 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-1c473fd0-b97b-40e0-a8c0-c05d3d260e1d None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:32.577253 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1c473fd0-b97b-40e0-a8c0-c05d3d260e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:32.604604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d837174-4350-4922-abe8-79fecc2aec2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:32.605271 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d837174-4350-4922-abe8-79fecc2aec2d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:32.605550 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d837174-4350-4922-abe8-79fecc2aec2d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:32.605878 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d837174-4350-4922-abe8-79fecc2aec2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:32.606794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d837174-4350-4922-abe8-79fecc2aec2d None admin] RBAC: Adding query filter params (role.id=343337c6f25943a9ad412047b70ddefa, scope.project.id=d95eae40782d41759de176dea214eb16, user.id=f25a34bc71ea41118a7a7546903e977e) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:32.607138 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d837174-4350-4922-abe8-79fecc2aec2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:32.608388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:32.608641 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d837174-4350-4922-abe8-79fecc2aec2d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:32.614930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 109/223] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:14:32 2022] GET /identity/v3/role_assignments?user.id=f25a34bc71ea41118a7a7546903e977e&scope.project.id=d95eae40782d41759de176dea214eb16&role.id=343337c6f25943a9ad412047b70ddefa => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:32.615754 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d837174-4350-4922-abe8-79fecc2aec2d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:32.649196 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 115/224] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:32.672503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c473fd0-b97b-40e0-a8c0-c05d3d260e1d None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:32.789845 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c473fd0-b97b-40e0-a8c0-c05d3d260e1d None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:32.832825 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732f234-577f-4718-87c0-e514a6cf087e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:32.833375 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d837174-4350-4922-abe8-79fecc2aec2d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:32.833629 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732f234-577f-4718-87c0-e514a6cf087e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:32.833909 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732f234-577f-4718-87c0-e514a6cf087e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:32.834181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732f234-577f-4718-87c0-e514a6cf087e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:32.850453 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d732f234-577f-4718-87c0-e514a6cf087e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:32.853505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:32.861061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-d732f234-577f-4718-87c0-e514a6cf087e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:14:32.861061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:32.861061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:32.861061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:32.861061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:32.861061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:32.861061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:32.861061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:32.861061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:32.861061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:32.861061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:32.861061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:32.861061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:32.861061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:32.861061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:32.861061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:32.862297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:32.862297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:32.862297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:32.862297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:32.862297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:32.862297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:32.862297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:32.862297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:32.862297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:32.862297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:32.862297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:32.862297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:14:32.862297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:32.862297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:32.862297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:32.862297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:32.863379 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:32.863379 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:32.863379 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:32.863379 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:32.863379 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:32.863379 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:32.863379 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:32.863379 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:32.863379 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:32.863379 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:32.863379 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:14:32.863379 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:14:32.863379 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 116/225] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:14:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:14:32.868664 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732f234-577f-4718-87c0-e514a6cf087e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:32.883674 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf9ca9-ac38-49f6-bdc6-172a43a0ccd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:32.884580 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf9ca9-ac38-49f6-bdc6-172a43a0ccd2 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:32.884963 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf9ca9-ac38-49f6-bdc6-172a43a0ccd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:32.885334 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf9ca9-ac38-49f6-bdc6-172a43a0ccd2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:32.904329 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bf9ca9-ac38-49f6-bdc6-172a43a0ccd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:32.907204 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:32.912936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-07bf9ca9-ac38-49f6-bdc6-172a43a0ccd2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:14:32.912936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:32.912936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:32.912936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:32.912936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:32.912936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:32.912936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:32.912936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:32.912936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:32.912936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:32.912936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:32.912936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:32.912936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:32.912936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:32.912936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:32.912936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:32.913812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:32.913812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:32.913812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:32.913812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:32.913812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:32.913812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:32.913812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:32.913812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:32.913812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:32.913812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:32.913812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:32.913812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:14:32.913812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:32.913812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:32.913812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:32.913812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:32.914588 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:32.914588 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:32.914588 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:32.914588 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:32.914588 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:32.914588 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:32.914588 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:32.914588 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:32.914588 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:32.914588 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:32.914588 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:14:32.914588 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:14:32.915225 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 110/226] 10.222.0.24 () {56 vars in 1114 bytes} [Fri May 27 03:14:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:32.921882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf9ca9-ac38-49f6-bdc6-172a43a0ccd2 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:33.011771 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732f234-577f-4718-87c0-e514a6cf087e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:33.062447 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384fa66-1421-4477-ac17-47b25861b550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:33.063211 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384fa66-1421-4477-ac17-47b25861b550 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:33.063491 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384fa66-1421-4477-ac17-47b25861b550 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:33.063762 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384fa66-1421-4477-ac17-47b25861b550 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:33.064518 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3384fa66-1421-4477-ac17-47b25861b550 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:33.064821 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3384fa66-1421-4477-ac17-47b25861b550 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:33.066211 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3384fa66-1421-4477-ac17-47b25861b550 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:33.087229 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 117/227] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:33.093740 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384fa66-1421-4477-ac17-47b25861b550 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:33.136054 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf9ca9-ac38-49f6-bdc6-172a43a0ccd2 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:33.205716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b8e45-493b-4787-884c-65ded830ec63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:33.206397 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b8e45-493b-4787-884c-65ded830ec63 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:33.206684 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b8e45-493b-4787-884c-65ded830ec63 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:33.206990 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b8e45-493b-4787-884c-65ded830ec63 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:33.207863 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b8e45-493b-4787-884c-65ded830ec63 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:33.208176 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b8e45-493b-4787-884c-65ded830ec63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:33.209704 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b8e45-493b-4787-884c-65ded830ec63 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:33.223167 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:33.228005 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 111/228] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:14:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:33.232198 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b8e45-493b-4787-884c-65ded830ec63 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:33.237335 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384fa66-1421-4477-ac17-47b25861b550 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:33.267996 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:33.308332 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e179d0-c968-4dd0-a895-8fe61b9d6463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:33.309289 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e179d0-c968-4dd0-a895-8fe61b9d6463 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:33.309643 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e179d0-c968-4dd0-a895-8fe61b9d6463 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:33.309923 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e179d0-c968-4dd0-a895-8fe61b9d6463 None admin] PATH_INFO: `/v3/users/f27304a61be64f9e8cab970a5298641f` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:33.314364 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e179d0-c968-4dd0-a895-8fe61b9d6463 None admin] RBAC: Authorizing `identity:get_user(user_id=f27304a61be64f9e8cab970a5298641f)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:33.315789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e179d0-c968-4dd0-a895-8fe61b9d6463 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:33.317442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 118/229] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:33 2022] GET /identity/v3/users/f27304a61be64f9e8cab970a5298641f => generated 283 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:33.324083 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e179d0-c968-4dd0-a895-8fe61b9d6463 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:33.342321 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b8e45-493b-4787-884c-65ded830ec63 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:33.378136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ded6fb-21a9-4974-b14a-b5a9bba0505c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:33.379367 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ded6fb-21a9-4974-b14a-b5a9bba0505c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:33.379843 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ded6fb-21a9-4974-b14a-b5a9bba0505c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:33.380305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ded6fb-21a9-4974-b14a-b5a9bba0505c None admin] PATH_INFO: `/v3/users/f27304a61be64f9e8cab970a5298641f` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:33.381732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ded6fb-21a9-4974-b14a-b5a9bba0505c None admin] RBAC: Authorizing `identity:get_user(user_id=f27304a61be64f9e8cab970a5298641f)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:33.383426 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ded6fb-21a9-4974-b14a-b5a9bba0505c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:33.385267 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 112/230] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:33 2022] GET /identity/v3/users/f27304a61be64f9e8cab970a5298641f => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:33.392856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ded6fb-21a9-4974-b14a-b5a9bba0505c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:33.483278 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e179d0-c968-4dd0-a895-8fe61b9d6463 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:33.548360 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b65419-d665-4ea5-ab0a-3298a1dfc08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:33.549244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b65419-d665-4ea5-ab0a-3298a1dfc08f None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:33.549523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b65419-d665-4ea5-ab0a-3298a1dfc08f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:33.549794 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b65419-d665-4ea5-ab0a-3298a1dfc08f None admin] PATH_INFO: `/v3/projects/d95eae40782d41759de176dea214eb16` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:33.551600 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b65419-d665-4ea5-ab0a-3298a1dfc08f None admin] RBAC: Authorizing `identity:get_project(project_id=d95eae40782d41759de176dea214eb16)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:33.553074 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b65419-d665-4ea5-ab0a-3298a1dfc08f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:33.554802 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 119/231] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:33 2022] GET /identity/v3/projects/d95eae40782d41759de176dea214eb16 => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:33.561613 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b65419-d665-4ea5-ab0a-3298a1dfc08f None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:33.584937 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ded6fb-21a9-4974-b14a-b5a9bba0505c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:33.587408 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:33.594358 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:33.674677 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b65419-d665-4ea5-ab0a-3298a1dfc08f None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:33.710151 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc0c944-f136-473e-94c6-dc92592d9d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:33.711325 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0c944-f136-473e-94c6-dc92592d9d60 None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:33.711529 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19579a-0720-42e5-acd6-9c46f12964dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:33.711912 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0c944-f136-473e-94c6-dc92592d9d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:33.712352 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc0c944-f136-473e-94c6-dc92592d9d60 None admin] PATH_INFO: `/v3/projects/d95eae40782d41759de176dea214eb16/users/f27304a61be64f9e8cab970a5298641f/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:33.712463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19579a-0720-42e5-acd6-9c46f12964dd None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:33.712687 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19579a-0720-42e5-acd6-9c46f12964dd None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:33.712972 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19579a-0720-42e5-acd6-9c46f12964dd None admin] PATH_INFO: `/v3/projects/d95eae40782d41759de176dea214eb16` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:33.715010 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc0c944-f136-473e-94c6-dc92592d9d60 None admin] RBAC: Authorizing `identity:create_grant(project_id=d95eae40782d41759de176dea214eb16, user_id=f27304a61be64f9e8cab970a5298641f, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:33.717204 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc0c944-f136-473e-94c6-dc92592d9d60 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:33.719555 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19579a-0720-42e5-acd6-9c46f12964dd None admin] RBAC: Authorizing `identity:get_project(project_id=d95eae40782d41759de176dea214eb16)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:33.721030 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa19579a-0720-42e5-acd6-9c46f12964dd None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:33.722967 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 113/232] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:33 2022] GET /identity/v3/projects/d95eae40782d41759de176dea214eb16 => generated 302 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:33.734876 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19579a-0720-42e5-acd6-9c46f12964dd None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:33.797650 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 120/233] 10.222.0.24 () {58 vars in 1439 bytes} [Fri May 27 03:14:33 2022] PUT /identity/v3/projects/d95eae40782d41759de176dea214eb16/users/f27304a61be64f9e8cab970a5298641f/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:14:33.847831 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19579a-0720-42e5-acd6-9c46f12964dd None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:33.890956 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e09a2-a3c6-4c1c-9114-d6895f340182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:33.892098 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e09a2-a3c6-4c1c-9114-d6895f340182 None admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:33.892547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e09a2-a3c6-4c1c-9114-d6895f340182 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:33.892961 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e09a2-a3c6-4c1c-9114-d6895f340182 None admin] PATH_INFO: `/v3/projects/d95eae40782d41759de176dea214eb16/users/f27304a61be64f9e8cab970a5298641f/roles/d7133ce917f0431883775551ab086d5d` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:33.900872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e09a2-a3c6-4c1c-9114-d6895f340182 None admin] RBAC: Authorizing `identity:create_grant(project_id=d95eae40782d41759de176dea214eb16, user_id=f27304a61be64f9e8cab970a5298641f, role_id=d7133ce917f0431883775551ab086d5d)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:33.902571 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e09a2-a3c6-4c1c-9114-d6895f340182 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:33.997496 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 114/234] 10.222.0.24 () {58 vars in 1439 bytes} [Fri May 27 03:14:33 2022] PUT /identity/v3/projects/d95eae40782d41759de176dea214eb16/users/f27304a61be64f9e8cab970a5298641f/roles/d7133ce917f0431883775551ab086d5d => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:14:34.545580 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8dadf-5a71-42c9-9f93-ad575788fffa None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:34.546102 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8dadf-5a71-42c9-9f93-ad575788fffa None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:34.546410 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8dadf-5a71-42c9-9f93-ad575788fffa None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:34.548169 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 121/235] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:34.553493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb42d4-90e0-41aa-a100-572a3145db07 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:34.553884 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb42d4-90e0-41aa-a100-572a3145db07 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:34.554282 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eb42d4-90e0-41aa-a100-572a3145db07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:34.580426 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-86eb42d4-90e0-41aa-a100-572a3145db07 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:34.581682 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-86eb42d4-90e0-41aa-a100-572a3145db07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:34.611048 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:34.661474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 115/236] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:34.701823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d8dadf-5a71-42c9-9f93-ad575788fffa None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:34.713221 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6becafd-cdc4-4eed-8433-ea01c1e4e696 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:34.713513 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6becafd-cdc4-4eed-8433-ea01c1e4e696 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:34.713795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6becafd-cdc4-4eed-8433-ea01c1e4e696 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:34.715481 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 116/237] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:34.720536 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a5606-f78e-4a7e-981c-dd45eacfe4d0 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:34.720824 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a5606-f78e-4a7e-981c-dd45eacfe4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:34.721107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a5606-f78e-4a7e-981c-dd45eacfe4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:34.739014 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d52a5606-f78e-4a7e-981c-dd45eacfe4d0 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:34.740221 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d52a5606-f78e-4a7e-981c-dd45eacfe4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:34.844185 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 117/238] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:14:34.860281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:34.879565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d8dadf-5a71-42c9-9f93-ad575788fffa None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:34.892150 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a5606-f78e-4a7e-981c-dd45eacfe4d0 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:34.949872 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db13c7c2-fecb-4794-9790-7f92ab770a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:34.951486 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13c7c2-fecb-4794-9790-7f92ab770a2a None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:34.951776 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13c7c2-fecb-4794-9790-7f92ab770a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:34.952052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db13c7c2-fecb-4794-9790-7f92ab770a2a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:34.964275 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db13c7c2-fecb-4794-9790-7f92ab770a2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:34.974858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-db13c7c2-fecb-4794-9790-7f92ab770a2a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:14:34.974858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:34.974858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:34.974858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:34.974858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:34.974858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:34.974858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:34.974858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:34.974858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:34.974858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:34.974858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:34.974858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:34.974858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:34.974858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:34.974858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:34.974858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:34.975762 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:34.975762 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:34.975762 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:34.975762 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:34.975762 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:34.975762 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:34.975762 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:34.975762 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:34.975762 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:34.975762 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:34.975762 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:34.975762 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:14:34.975762 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:34.975762 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:34.975762 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:34.975762 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:34.976541 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:34.976541 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:34.976541 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:34.976541 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:34.976541 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:34.976541 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:34.976541 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:34.976541 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:34.976541 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:34.976541 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:34.976541 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:14:34.976541 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:14:34.977123 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 122/239] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:34.981771 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db13c7c2-fecb-4794-9790-7f92ab770a2a None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:35.067938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:35.091464 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a5606-f78e-4a7e-981c-dd45eacfe4d0 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:35.152953 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db13c7c2-fecb-4794-9790-7f92ab770a2a None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:35.159418 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d801d-d5d3-4c10-b82d-a2b493c62623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:35.160401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d801d-d5d3-4c10-b82d-a2b493c62623 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:35.160820 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d801d-d5d3-4c10-b82d-a2b493c62623 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:35.161248 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d801d-d5d3-4c10-b82d-a2b493c62623 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:35.162544 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d801d-d5d3-4c10-b82d-a2b493c62623 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:35.171771 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d801d-d5d3-4c10-b82d-a2b493c62623 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:35.173647 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 118/240] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:14:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:35.180120 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d801d-d5d3-4c10-b82d-a2b493c62623 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:35.193197 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79432cbb-f782-4a58-9f2c-4a2b37b1d9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:35.193876 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79432cbb-f782-4a58-9f2c-4a2b37b1d9d6 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:35.194162 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79432cbb-f782-4a58-9f2c-4a2b37b1d9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:35.194438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79432cbb-f782-4a58-9f2c-4a2b37b1d9d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:35.195237 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79432cbb-f782-4a58-9f2c-4a2b37b1d9d6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:35.195545 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79432cbb-f782-4a58-9f2c-4a2b37b1d9d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:35.196941 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79432cbb-f782-4a58-9f2c-4a2b37b1d9d6 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:35.207264 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 123/241] 10.222.0.24 () {56 vars in 1101 bytes} [Fri May 27 03:14:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:35.218394 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79432cbb-f782-4a58-9f2c-4a2b37b1d9d6 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:35.239166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:35.304000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d801d-d5d3-4c10-b82d-a2b493c62623 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:35.340535 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662a5f9-5785-4799-9655-3a552498078a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:35.341609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662a5f9-5785-4799-9655-3a552498078a None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:35.342083 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662a5f9-5785-4799-9655-3a552498078a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:35.342546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662a5f9-5785-4799-9655-3a552498078a None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:35.351564 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662a5f9-5785-4799-9655-3a552498078a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:35.353306 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662a5f9-5785-4799-9655-3a552498078a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:35.385596 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79432cbb-f782-4a58-9f2c-4a2b37b1d9d6 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:35.421318 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30cb26-3b28-472d-a65b-ada04814a952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:35.422127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30cb26-3b28-472d-a65b-ada04814a952 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:35.422414 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30cb26-3b28-472d-a65b-ada04814a952 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:35.422689 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30cb26-3b28-472d-a65b-ada04814a952 None admin] PATH_INFO: `/v3/users/5ba57ca07c704b99bc61f73a3c129cdc` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:35.436998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 119/242] 10.222.0.24 () {60 vars in 1132 bytes} [Fri May 27 03:14:35 2022] POST /identity/v3/users => generated 293 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:14:35.438787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c507d1-a7d9-4d81-b8bf-b6110bd5bad3 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:35.439408 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c507d1-a7d9-4d81-b8bf-b6110bd5bad3 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:35.439872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c507d1-a7d9-4d81-b8bf-b6110bd5bad3 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:35.447414 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30cb26-3b28-472d-a65b-ada04814a952 None admin] RBAC: Authorizing `identity:get_user(user_id=5ba57ca07c704b99bc61f73a3c129cdc)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:35.447414 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30cb26-3b28-472d-a65b-ada04814a952 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:35.447609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 120/243] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:35.447609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6aba42-2d50-4600-ae0c-c08908d794c5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:35.447731 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 124/244] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:35 2022] GET /identity/v3/users/5ba57ca07c704b99bc61f73a3c129cdc => generated 297 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:35.448082 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6aba42-2d50-4600-ae0c-c08908d794c5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:35.448530 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6aba42-2d50-4600-ae0c-c08908d794c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:35.457680 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30cb26-3b28-472d-a65b-ada04814a952 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:35.476844 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-fe6aba42-2d50-4600-ae0c-c08908d794c5 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:35.478821 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-fe6aba42-2d50-4600-ae0c-c08908d794c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:35.492500 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:35.604528 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 121/245] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:14:35.618311 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:35.672901 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30cb26-3b28-472d-a65b-ada04814a952 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:35.677799 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6aba42-2d50-4600-ae0c-c08908d794c5 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:35.712924 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088cea7-2c94-4cb3-ab18-72b85f1f1e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:35.713584 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088cea7-2c94-4cb3-ab18-72b85f1f1e36 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:35.713870 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088cea7-2c94-4cb3-ab18-72b85f1f1e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:35.714221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8088cea7-2c94-4cb3-ab18-72b85f1f1e36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:35.798177 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6aba42-2d50-4600-ae0c-c08908d794c5 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:35.801317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8088cea7-2c94-4cb3-ab18-72b85f1f1e36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:35.801317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8088cea7-2c94-4cb3-ab18-72b85f1f1e36 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:35.842811 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db268483-6608-4cf1-8991-67915412254a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:35.844073 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db268483-6608-4cf1-8991-67915412254a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:35.844620 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db268483-6608-4cf1-8991-67915412254a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:35.845096 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db268483-6608-4cf1-8991-67915412254a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:35.859900 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db268483-6608-4cf1-8991-67915412254a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:35.868378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-db268483-6608-4cf1-8991-67915412254a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:35.868378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:35.868378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:35.868378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:35.868378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:35.868378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:35.868378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:35.868378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:35.868378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:35.868378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:35.868378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:35.868378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:35.868378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:35.868378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:35.868378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:35.868378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:35.869315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:35.869315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:35.869315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:35.869315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:35.869315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:35.869315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:35.869315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:35.869315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:35.869315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:35.869315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:35.869315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:35.869315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:14:35.869315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:35.869315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:35.869315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:35.869315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:35.870089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:35.870089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:35.870089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:35.870089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:35.870089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:35.870089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:35.870089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:35.870089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:35.870089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:35.870089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:35.870089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:35.870089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:14:35.873551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 122/246] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:35.879965 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:35.880578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db268483-6608-4cf1-8991-67915412254a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:35.911223 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:35.926577 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 125/247] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:14:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:14:35.932680 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088cea7-2c94-4cb3-ab18-72b85f1f1e36 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:36.084956 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db268483-6608-4cf1-8991-67915412254a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:36.125278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d8abb-f422-4262-8021-a94a2863dbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:36.125985 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8088cea7-2c94-4cb3-ab18-72b85f1f1e36 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:36.126505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d8abb-f422-4262-8021-a94a2863dbf3 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:36.127027 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d8abb-f422-4262-8021-a94a2863dbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:36.127583 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d8abb-f422-4262-8021-a94a2863dbf3 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:36.128646 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d8abb-f422-4262-8021-a94a2863dbf3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:36.129154 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d8abb-f422-4262-8021-a94a2863dbf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:36.130749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d8abb-f422-4262-8021-a94a2863dbf3 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:36.142509 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 123/248] 10.222.0.24 () {56 vars in 1101 bytes} [Fri May 27 03:14:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:36.148638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d8abb-f422-4262-8021-a94a2863dbf3 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:36.167896 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d2eee3-806a-4389-aa7f-861a2a94d61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:36.169314 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2eee3-806a-4389-aa7f-861a2a94d61f None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:36.169599 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2eee3-806a-4389-aa7f-861a2a94d61f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:36.169874 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2eee3-806a-4389-aa7f-861a2a94d61f None admin] PATH_INFO: `/v3/projects/d95eae40782d41759de176dea214eb16` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:36.171787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d2eee3-806a-4389-aa7f-861a2a94d61f None admin] RBAC: Authorizing `identity:get_project(project_id=d95eae40782d41759de176dea214eb16)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:36.173265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d2eee3-806a-4389-aa7f-861a2a94d61f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:36.175172 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 126/249] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:35 2022] GET /identity/v3/projects/d95eae40782d41759de176dea214eb16 => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:36.180744 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d2eee3-806a-4389-aa7f-861a2a94d61f None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:36.218902 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:36.228385 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:36.268417 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d8abb-f422-4262-8021-a94a2863dbf3 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:36.294339 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d2eee3-806a-4389-aa7f-861a2a94d61f None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:36.304878 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d85a3c-e99d-4f99-b0d4-0dbcf3980d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:36.306084 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d85a3c-e99d-4f99-b0d4-0dbcf3980d22 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:36.306606 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d85a3c-e99d-4f99-b0d4-0dbcf3980d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:36.307116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d85a3c-e99d-4f99-b0d4-0dbcf3980d22 None admin] PATH_INFO: `/v3/users/f25a34bc71ea41118a7a7546903e977e` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:36.308508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d85a3c-e99d-4f99-b0d4-0dbcf3980d22 None admin] RBAC: Authorizing `identity:get_user(user_id=f25a34bc71ea41118a7a7546903e977e)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:36.310727 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d85a3c-e99d-4f99-b0d4-0dbcf3980d22 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:36.312874 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 124/250] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:36 2022] GET /identity/v3/users/f25a34bc71ea41118a7a7546903e977e => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:36.318370 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d85a3c-e99d-4f99-b0d4-0dbcf3980d22 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:36.332076 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61bc83-7074-47aa-bd83-d1aece98167f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:36.332748 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61bc83-7074-47aa-bd83-d1aece98167f None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:36.333125 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61bc83-7074-47aa-bd83-d1aece98167f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:36.333294 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa61bc83-7074-47aa-bd83-d1aece98167f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:36.334128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa61bc83-7074-47aa-bd83-d1aece98167f None admin] RBAC: Adding query filter params (role.id=aa0d3c6b36a24a718f88d0f4309d26d9, scope.project.id=d95eae40782d41759de176dea214eb16, user.id=5ba57ca07c704b99bc61f73a3c129cdc) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:36.334442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa61bc83-7074-47aa-bd83-d1aece98167f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:36.335924 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa61bc83-7074-47aa-bd83-d1aece98167f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:36.341835 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 127/251] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:14:36 2022] GET /identity/v3/role_assignments?user.id=5ba57ca07c704b99bc61f73a3c129cdc&scope.project.id=d95eae40782d41759de176dea214eb16&role.id=aa0d3c6b36a24a718f88d0f4309d26d9 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:36.432885 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d85a3c-e99d-4f99-b0d4-0dbcf3980d22 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:36.473940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e8d55-f0f0-4389-84d7-6213524b6c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:36.475254 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e8d55-f0f0-4389-84d7-6213524b6c7e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:36.475737 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e8d55-f0f0-4389-84d7-6213524b6c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:36.476261 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e8d55-f0f0-4389-84d7-6213524b6c7e None admin] PATH_INFO: `/v3/projects/d95eae40782d41759de176dea214eb16` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:36.477818 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e8d55-f0f0-4389-84d7-6213524b6c7e None admin] RBAC: Authorizing `identity:get_project(project_id=d95eae40782d41759de176dea214eb16)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:36.479624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e8d55-f0f0-4389-84d7-6213524b6c7e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:36.481705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 125/252] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:36 2022] GET /identity/v3/projects/d95eae40782d41759de176dea214eb16 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:36.487516 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61bc83-7074-47aa-bd83-d1aece98167f None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:36.609210 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:36.613591 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa61bc83-7074-47aa-bd83-d1aece98167f None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:36.656727 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef5ade-adec-4f20-be73-034b89695953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:36.658536 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef5ade-adec-4f20-be73-034b89695953 None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:36.659081 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef5ade-adec-4f20-be73-034b89695953 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:36.659547 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef5ade-adec-4f20-be73-034b89695953 None admin] PATH_INFO: `/v3/projects/d95eae40782d41759de176dea214eb16/users/f25a34bc71ea41118a7a7546903e977e/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:36.661902 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef5ade-adec-4f20-be73-034b89695953 None admin] RBAC: Authorizing `identity:create_grant(project_id=d95eae40782d41759de176dea214eb16, user_id=f25a34bc71ea41118a7a7546903e977e, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:36.663614 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef5ade-adec-4f20-be73-034b89695953 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:36.694992 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6422c-eba5-4415-a327-3de173a4bc2c None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:36.695476 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6422c-eba5-4415-a327-3de173a4bc2c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:36.695476 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6422c-eba5-4415-a327-3de173a4bc2c None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:36.696821 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 126/253] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:14:36.702745 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f28aa-dbd4-4dd0-9a9e-0b732b0964f8 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:36.703071 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f28aa-dbd4-4dd0-9a9e-0b732b0964f8 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:36.703332 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f28aa-dbd4-4dd0-9a9e-0b732b0964f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:36.709134 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:36.723277 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-122f28aa-dbd4-4dd0-9a9e-0b732b0964f8 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:36.724807 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-122f28aa-dbd4-4dd0-9a9e-0b732b0964f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:36.754136 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 128/254] 10.222.0.24 () {58 vars in 1439 bytes} [Fri May 27 03:14:36 2022] PUT /identity/v3/projects/d95eae40782d41759de176dea214eb16/users/f25a34bc71ea41118a7a7546903e977e/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:14:36.810926 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 127/255] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:14:36.836923 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef5ade-adec-4f20-be73-034b89695953 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:36.899234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5a128-153c-49ff-a632-cf7319416d4a None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:36.899517 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5a128-153c-49ff-a632-cf7319416d4a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:36.899842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5a128-153c-49ff-a632-cf7319416d4a None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:36.901466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 128/256] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:36.907226 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8202ba-8a98-439d-8299-da2845f93922 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:36.907509 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8202ba-8a98-439d-8299-da2845f93922 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:36.907786 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8202ba-8a98-439d-8299-da2845f93922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:36.925859 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-6e8202ba-8a98-439d-8299-da2845f93922 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:36.927217 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6e8202ba-8a98-439d-8299-da2845f93922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:37.014400 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef5ade-adec-4f20-be73-034b89695953 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:37.048622 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 129/257] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:14:37.091906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8202ba-8a98-439d-8299-da2845f93922 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:37.099583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff479e-4672-4ce4-922c-a46ae26fde9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:37.100407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff479e-4672-4ce4-922c-a46ae26fde9f None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:37.100692 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff479e-4672-4ce4-922c-a46ae26fde9f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:37.100970 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ff479e-4672-4ce4-922c-a46ae26fde9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:37.138405 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ff479e-4672-4ce4-922c-a46ae26fde9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:37.146517 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-a7ff479e-4672-4ce4-922c-a46ae26fde9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:14:37.146517 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:37.146517 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:37.146517 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:37.146517 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:37.146517 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:37.146517 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:37.146517 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:37.146517 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:37.146517 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:37.146517 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:37.146517 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:37.146517 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:37.146517 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:37.146517 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:37.146517 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:37.147387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:37.147387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:37.147387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:37.147387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:37.147387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:37.147387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:37.147387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:37.147387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:37.147387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:37.147387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:37.147387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:37.147387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:14:37.147387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:37.147387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:37.147387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:37.147387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:37.148178 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:37.148178 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:37.148178 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:37.148178 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:37.148178 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:37.148178 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:37.148178 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:37.148178 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:37.148178 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:37.148178 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:37.148178 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:14:37.148178 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:14:37.148178 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 129/258] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:14:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:37.154006 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff479e-4672-4ce4-922c-a46ae26fde9f None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:37.176926 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:37.205953 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:37.272659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8202ba-8a98-439d-8299-da2845f93922 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:37.310891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ff479e-4672-4ce4-922c-a46ae26fde9f None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:37.331528 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce7d00-bed5-4357-b13b-2900d47772da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:37.332697 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce7d00-bed5-4357-b13b-2900d47772da None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:37.333175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce7d00-bed5-4357-b13b-2900d47772da None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:37.333663 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcce7d00-bed5-4357-b13b-2900d47772da None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:37.354222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcce7d00-bed5-4357-b13b-2900d47772da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:37.370856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-fcce7d00-bed5-4357-b13b-2900d47772da None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:14:37.370856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:37.370856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:37.370856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:37.370856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:37.370856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:37.370856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:37.370856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:37.370856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:37.370856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:37.370856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:37.370856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:37.370856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:37.370856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:37.370856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:37.370856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:37.372285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:37.372285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:37.372285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:37.372285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:37.372285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:37.372285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:37.372285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:37.372285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:37.372285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:37.372285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:37.372285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:37.372285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:14:37.372285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:37.372285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:37.372285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:37.372285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:37.373060 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:37.373060 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:37.373060 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:37.373060 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:37.373060 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:37.373060 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:37.373060 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:37.373060 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:37.373060 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:37.373060 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:37.373060 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:14:37.373060 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:14:37.375844 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 130/259] 10.222.0.24 () {56 vars in 1114 bytes} [Fri May 27 03:14:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:14:37.381210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce7d00-bed5-4357-b13b-2900d47772da None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:37.384403 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21863b9c-4090-4853-aa30-ad0552566579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:37.385252 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21863b9c-4090-4853-aa30-ad0552566579 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:37.385474 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21863b9c-4090-4853-aa30-ad0552566579 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:37.385755 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21863b9c-4090-4853-aa30-ad0552566579 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:37.386516 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21863b9c-4090-4853-aa30-ad0552566579 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:37.387793 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21863b9c-4090-4853-aa30-ad0552566579 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:37.388402 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21863b9c-4090-4853-aa30-ad0552566579 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:37.408394 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 130/260] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:37.413531 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21863b9c-4090-4853-aa30-ad0552566579 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:37.487496 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcce7d00-bed5-4357-b13b-2900d47772da None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:37.511250 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:37.525822 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16fe82-a50f-457e-945a-26174c7bcd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:37.525822 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16fe82-a50f-457e-945a-26174c7bcd8c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:37.525822 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16fe82-a50f-457e-945a-26174c7bcd8c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:37.525822 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16fe82-a50f-457e-945a-26174c7bcd8c None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:37.526171 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed16fe82-a50f-457e-945a-26174c7bcd8c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:37.526618 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed16fe82-a50f-457e-945a-26174c7bcd8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:37.528811 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed16fe82-a50f-457e-945a-26174c7bcd8c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:37.538413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 131/261] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:14:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:37.544430 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16fe82-a50f-457e-945a-26174c7bcd8c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:37.547012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:37.575612 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21863b9c-4090-4853-aa30-ad0552566579 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:37.634683 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a38247-8d31-42f9-be48-a7d0d5dff97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:37.635615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a38247-8d31-42f9-be48-a7d0d5dff97b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:37.635908 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a38247-8d31-42f9-be48-a7d0d5dff97b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:37.636170 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a38247-8d31-42f9-be48-a7d0d5dff97b None admin] PATH_INFO: `/v3/users/f27304a61be64f9e8cab970a5298641f` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:37.638307 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a38247-8d31-42f9-be48-a7d0d5dff97b None admin] RBAC: Authorizing `identity:get_user(user_id=f27304a61be64f9e8cab970a5298641f)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:37.640160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a38247-8d31-42f9-be48-a7d0d5dff97b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:37.642495 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 131/262] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:37 2022] GET /identity/v3/users/f27304a61be64f9e8cab970a5298641f => generated 283 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:37.649319 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a38247-8d31-42f9-be48-a7d0d5dff97b None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:37.657827 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16fe82-a50f-457e-945a-26174c7bcd8c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:37.718291 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725ffb8-6e64-470f-b1a3-b9824d886ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:37.719361 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725ffb8-6e64-470f-b1a3-b9824d886ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:37.719728 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725ffb8-6e64-470f-b1a3-b9824d886ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:37.720096 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725ffb8-6e64-470f-b1a3-b9824d886ea0 None admin] PATH_INFO: `/v3/users/f27304a61be64f9e8cab970a5298641f` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:37.721389 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725ffb8-6e64-470f-b1a3-b9824d886ea0 None admin] RBAC: Authorizing `identity:get_user(user_id=f27304a61be64f9e8cab970a5298641f)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:37.722953 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725ffb8-6e64-470f-b1a3-b9824d886ea0 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:37.724675 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 132/263] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:37 2022] GET /identity/v3/users/f27304a61be64f9e8cab970a5298641f => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:37.730159 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725ffb8-6e64-470f-b1a3-b9824d886ea0 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:37.765837 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a38247-8d31-42f9-be48-a7d0d5dff97b None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:37.805265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3b1e0-4d2e-4593-827c-53f09c74dace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:37.806170 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3b1e0-4d2e-4593-827c-53f09c74dace None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:37.806594 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3b1e0-4d2e-4593-827c-53f09c74dace None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:37.807067 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3b1e0-4d2e-4593-827c-53f09c74dace None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:37.821498 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:37.827519 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:37.881152 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3b1e0-4d2e-4593-827c-53f09c74dace None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:37.882892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea3b1e0-4d2e-4593-827c-53f09c74dace None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:37.939743 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725ffb8-6e64-470f-b1a3-b9824d886ea0 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:38.010355 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 132/264] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:14:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:14:38.017659 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3b1e0-4d2e-4593-827c-53f09c74dace None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:38.063641 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64721198-e686-49ec-a21a-2bcc2ca6323a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:38.064325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64721198-e686-49ec-a21a-2bcc2ca6323a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:38.064506 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64721198-e686-49ec-a21a-2bcc2ca6323a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:38.064792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64721198-e686-49ec-a21a-2bcc2ca6323a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:38.083286 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:38.143689 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea3b1e0-4d2e-4593-827c-53f09c74dace None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:38.182352 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dffe8f-ddd8-49fe-984c-47809a813aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:38.183262 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dffe8f-ddd8-49fe-984c-47809a813aeb None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:38.183548 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dffe8f-ddd8-49fe-984c-47809a813aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:38.183822 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dffe8f-ddd8-49fe-984c-47809a813aeb None admin] PATH_INFO: `/v3/projects/d95eae40782d41759de176dea214eb16` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:38.185032 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dffe8f-ddd8-49fe-984c-47809a813aeb None admin] RBAC: Authorizing `identity:get_project(project_id=d95eae40782d41759de176dea214eb16)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:38.186472 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dffe8f-ddd8-49fe-984c-47809a813aeb None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:38.188492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 133/265] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:38 2022] GET /identity/v3/projects/d95eae40782d41759de176dea214eb16 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:38.195158 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dffe8f-ddd8-49fe-984c-47809a813aeb None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:38.287924 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64721198-e686-49ec-a21a-2bcc2ca6323a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:38.289445 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64721198-e686-49ec-a21a-2bcc2ca6323a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:38.300300 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dffe8f-ddd8-49fe-984c-47809a813aeb None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:38.325630 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:38.336453 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237b52f-2d18-4794-96cd-7a4d57afd0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:38.337124 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237b52f-2d18-4794-96cd-7a4d57afd0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:38.337408 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237b52f-2d18-4794-96cd-7a4d57afd0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:38.337810 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237b52f-2d18-4794-96cd-7a4d57afd0d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:38.338654 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237b52f-2d18-4794-96cd-7a4d57afd0d1 None admin] RBAC: Adding query filter params (role.id=e5a908bd3ed5445b82a6053246533b88, scope.project.id=d95eae40782d41759de176dea214eb16, user.id=f27304a61be64f9e8cab970a5298641f) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:38.338997 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237b52f-2d18-4794-96cd-7a4d57afd0d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:38.341047 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237b52f-2d18-4794-96cd-7a4d57afd0d1 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:38.347433 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 134/266] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:14:38 2022] GET /identity/v3/role_assignments?user.id=f27304a61be64f9e8cab970a5298641f&scope.project.id=d95eae40782d41759de176dea214eb16&role.id=e5a908bd3ed5445b82a6053246533b88 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:38.378701 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:38.449664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 133/267] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:14:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 720 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:14:38.456210 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237b52f-2d18-4794-96cd-7a4d57afd0d1 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:38.570748 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237b52f-2d18-4794-96cd-7a4d57afd0d1 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:38.609342 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95710919-8448-4ed8-9693-9c99837434e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:38.610138 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95710919-8448-4ed8-9693-9c99837434e8 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:38.610422 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95710919-8448-4ed8-9693-9c99837434e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:38.610693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95710919-8448-4ed8-9693-9c99837434e8 None admin] PATH_INFO: `/v3/projects/d95eae40782d41759de176dea214eb16` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:38.611959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95710919-8448-4ed8-9693-9c99837434e8 None admin] RBAC: Authorizing `identity:get_project(project_id=d95eae40782d41759de176dea214eb16)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:38.613394 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95710919-8448-4ed8-9693-9c99837434e8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:38.615297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 135/268] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:38 2022] GET /identity/v3/projects/d95eae40782d41759de176dea214eb16 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:38.620999 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64721198-e686-49ec-a21a-2bcc2ca6323a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:38.749192 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64721198-e686-49ec-a21a-2bcc2ca6323a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:38.795732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551fba0d-29a3-46f2-b1f2-ecdf1d300dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:38.796392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fba0d-29a3-46f2-b1f2-ecdf1d300dbb None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:38.796674 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fba0d-29a3-46f2-b1f2-ecdf1d300dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:38.796947 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551fba0d-29a3-46f2-b1f2-ecdf1d300dbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:38.797773 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fba0d-29a3-46f2-b1f2-ecdf1d300dbb None admin] RBAC: Adding query filter params (role.id=d7133ce917f0431883775551ab086d5d, scope.project.id=d95eae40782d41759de176dea214eb16, user.id=f27304a61be64f9e8cab970a5298641f) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:38.798085 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fba0d-29a3-46f2-b1f2-ecdf1d300dbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:38.799579 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551fba0d-29a3-46f2-b1f2-ecdf1d300dbb None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:38.805940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 134/269] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:14:38 2022] GET /identity/v3/role_assignments?user.id=f27304a61be64f9e8cab970a5298641f&scope.project.id=d95eae40782d41759de176dea214eb16&role.id=d7133ce917f0431883775551ab086d5d => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:39.248141 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf0319-32a9-45f7-bfd1-820d8da8f72d None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:39.248672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf0319-32a9-45f7-bfd1-820d8da8f72d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:39.248672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf0319-32a9-45f7-bfd1-820d8da8f72d None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:39.250316 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 136/270] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:39.255617 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae103a6-1470-4840-a0a7-18f4305c49c3 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:39.255906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae103a6-1470-4840-a0a7-18f4305c49c3 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:39.256184 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae103a6-1470-4840-a0a7-18f4305c49c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:39.275050 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-fae103a6-1470-4840-a0a7-18f4305c49c3 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:39.276257 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-fae103a6-1470-4840-a0a7-18f4305c49c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:39.291102 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:39.355486 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 135/271] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:39.383549 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf0319-32a9-45f7-bfd1-820d8da8f72d None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:39.397033 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba18e3d-4585-4650-9485-8b22f9a021ae None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:39.397327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba18e3d-4585-4650-9485-8b22f9a021ae None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:39.397599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba18e3d-4585-4650-9485-8b22f9a021ae None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:39.399114 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 136/272] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:39.404503 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77aacb-5c33-4991-9fab-d5e9dc160caa None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:39.404792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77aacb-5c33-4991-9fab-d5e9dc160caa None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:39.405073 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77aacb-5c33-4991-9fab-d5e9dc160caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:39.422738 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-de77aacb-5c33-4991-9fab-d5e9dc160caa None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:39.423955 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-de77aacb-5c33-4991-9fab-d5e9dc160caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:39.446268 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:39.500266 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 137/273] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:39.516871 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf0319-32a9-45f7-bfd1-820d8da8f72d None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:39.523001 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77aacb-5c33-4991-9fab-d5e9dc160caa None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:39.556910 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c9896-3984-4939-815f-7329d4e1dbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:39.557719 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c9896-3984-4939-815f-7329d4e1dbb7 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:39.557996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c9896-3984-4939-815f-7329d4e1dbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:39.558325 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c9896-3984-4939-815f-7329d4e1dbb7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:39.560147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9c9896-3984-4939-815f-7329d4e1dbb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:39.561660 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9c9896-3984-4939-815f-7329d4e1dbb7 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:39.563743 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 137/274] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:14:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:39.569167 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c9896-3984-4939-815f-7329d4e1dbb7 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:39.616403 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:39.640088 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77aacb-5c33-4991-9fab-d5e9dc160caa None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:39.684762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e220eb-b05f-455d-86bc-39f737c27a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:39.684762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e220eb-b05f-455d-86bc-39f737c27a08 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:39.684762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e220eb-b05f-455d-86bc-39f737c27a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:39.684762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e220eb-b05f-455d-86bc-39f737c27a08 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:39.696650 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e220eb-b05f-455d-86bc-39f737c27a08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:39.702937 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c9896-3984-4939-815f-7329d4e1dbb7 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:39.707740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-c9e220eb-b05f-455d-86bc-39f737c27a08 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:39.707740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:39.707740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:39.707740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:39.707740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:39.707740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:39.707740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:39.707740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:39.707740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:39.707740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:39.707740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:39.707740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:39.707740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:39.707740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:39.707740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:39.707740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:39.708870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:39.708870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:39.708870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:39.708870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:39.708870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:39.708870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:39.708870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:39.708870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:39.708870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:39.708870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:39.708870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:39.708870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:14:39.708870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:39.708870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:39.708870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:39.708870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:39.709658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:39.709658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:39.709658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:39.709658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:39.709658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:39.709658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:39.709658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:39.709658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:39.709658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:39.709658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:39.709658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:39.709658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:14:39.712402 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 138/275] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:39.718595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e220eb-b05f-455d-86bc-39f737c27a08 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:39.726044 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:39.751574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35267fa4-381d-44df-86a9-cd808050d0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:39.752943 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35267fa4-381d-44df-86a9-cd808050d0b7 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:39.753983 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35267fa4-381d-44df-86a9-cd808050d0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:39.754271 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35267fa4-381d-44df-86a9-cd808050d0b7 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:39.755178 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35267fa4-381d-44df-86a9-cd808050d0b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:39.756689 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35267fa4-381d-44df-86a9-cd808050d0b7 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:39.830487 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e220eb-b05f-455d-86bc-39f737c27a08 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:39.860210 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 138/276] 10.222.0.24 () {60 vars in 1132 bytes} [Fri May 27 03:14:39 2022] POST /identity/v3/users => generated 293 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:14:39.861639 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb8b04-ca84-4156-8a39-0ad6c718516c None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:39.862192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb8b04-ca84-4156-8a39-0ad6c718516c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:39.862472 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb8b04-ca84-4156-8a39-0ad6c718516c None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:39.863908 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 139/277] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:39.869741 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59292326-3233-4c08-8648-59afdb4c9cd4 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:39.869985 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59292326-3233-4c08-8648-59afdb4c9cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:39.870213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59292326-3233-4c08-8648-59afdb4c9cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:39.870530 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3afc5-fb59-4b11-8820-319ee03d6bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:39.871240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3afc5-fb59-4b11-8820-319ee03d6bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:39.871699 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3afc5-fb59-4b11-8820-319ee03d6bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:39.871799 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d3afc5-fb59-4b11-8820-319ee03d6bd2 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:39.872537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d3afc5-fb59-4b11-8820-319ee03d6bd2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:39.872907 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d3afc5-fb59-4b11-8820-319ee03d6bd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:39.874362 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d3afc5-fb59-4b11-8820-319ee03d6bd2 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:39.877619 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:39.890639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 139/278] 10.222.0.24 () {56 vars in 1101 bytes} [Fri May 27 03:14:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:39.891973 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3afc5-fb59-4b11-8820-319ee03d6bd2 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:39.908453 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-59292326-3233-4c08-8648-59afdb4c9cd4 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:39.909684 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-59292326-3233-4c08-8648-59afdb4c9cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:39.999000 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 140/279] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:40.000608 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae925b-7132-4d37-b19d-9a5fc9de74b4 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:40.000890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae925b-7132-4d37-b19d-9a5fc9de74b4 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:40.001164 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae925b-7132-4d37-b19d-9a5fc9de74b4 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:40.002797 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 141/280] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:40.008688 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e06cd-2c19-468a-af75-3e16efd94455 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:40.009013 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e06cd-2c19-468a-af75-3e16efd94455 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:40.009241 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e06cd-2c19-468a-af75-3e16efd94455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:40.028926 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-4d3e06cd-2c19-468a-af75-3e16efd94455 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:40.030142 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4d3e06cd-2c19-468a-af75-3e16efd94455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:40.030413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d3afc5-fb59-4b11-8820-319ee03d6bd2 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:40.072548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d584ac2-f9dc-473d-a773-0fee8c86aec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:40.073430 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d584ac2-f9dc-473d-a773-0fee8c86aec0 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:40.073715 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d584ac2-f9dc-473d-a773-0fee8c86aec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:40.073989 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d584ac2-f9dc-473d-a773-0fee8c86aec0 None admin] PATH_INFO: `/v3/users/21c88ba21e1141efafca737507619cb9` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:40.095143 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d584ac2-f9dc-473d-a773-0fee8c86aec0 None admin] RBAC: Authorizing `identity:get_user(user_id=21c88ba21e1141efafca737507619cb9)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:40.095438 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d584ac2-f9dc-473d-a773-0fee8c86aec0 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:40.097119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 140/281] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:39 2022] GET /identity/v3/users/21c88ba21e1141efafca737507619cb9 => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:40.097966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d584ac2-f9dc-473d-a773-0fee8c86aec0 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:40.117713 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 142/282] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:14:40.118559 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e06cd-2c19-468a-af75-3e16efd94455 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:40.161078 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:40.163797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:40.218051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d584ac2-f9dc-473d-a773-0fee8c86aec0 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:40.238214 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e06cd-2c19-468a-af75-3e16efd94455 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:40.259007 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996416f-ac49-44cc-90f0-740b748b314b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:40.259952 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996416f-ac49-44cc-90f0-740b748b314b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:40.260238 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996416f-ac49-44cc-90f0-740b748b314b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:40.260507 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996416f-ac49-44cc-90f0-740b748b314b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:40.279190 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6996416f-ac49-44cc-90f0-740b748b314b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:40.283035 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ade21c-0575-4419-9d96-3c9a614df517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:40.283694 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ade21c-0575-4419-9d96-3c9a614df517 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:40.283976 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ade21c-0575-4419-9d96-3c9a614df517 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:40.284249 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ade21c-0575-4419-9d96-3c9a614df517 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:40.285081 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ade21c-0575-4419-9d96-3c9a614df517 None admin] RBAC: Adding query filter params (role.id=343337c6f25943a9ad412047b70ddefa, user.id=21c88ba21e1141efafca737507619cb9, scope.system=all) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:40.285389 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ade21c-0575-4419-9d96-3c9a614df517 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:40.286824 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ade21c-0575-4419-9d96-3c9a614df517 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:40.287320 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-6996416f-ac49-44cc-90f0-740b748b314b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:14:40.287320 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:40.287320 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:40.287320 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:40.287320 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:40.287320 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:40.287320 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:40.287320 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:40.287320 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:40.287320 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:40.287320 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:40.287320 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:40.287320 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:40.287320 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:40.287320 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:40.287320 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:40.288180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:40.288180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:40.288180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:40.288180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:40.288180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:40.288180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:40.288180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:40.288180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:40.288180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:40.288180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:40.288180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:40.288180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:14:40.288180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:40.288180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:40.288180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:40.288180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:40.288966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:40.288966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:40.288966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:40.288966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:40.288966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:40.288966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:40.288966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:40.288966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:40.288966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:40.288966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:40.288966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:14:40.288966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:14:40.290029 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 141/283] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:40.290864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996416f-ac49-44cc-90f0-740b748b314b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:40.296461 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 143/284] 10.222.0.24 () {56 vars in 1308 bytes} [Fri May 27 03:14:40 2022] GET /identity/v3/role_assignments?user.id=21c88ba21e1141efafca737507619cb9&scope.system=all&role.id=343337c6f25943a9ad412047b70ddefa => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:40.297488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ade21c-0575-4419-9d96-3c9a614df517 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:40.490339 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996416f-ac49-44cc-90f0-740b748b314b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:40.504218 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:40.511823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ade21c-0575-4419-9d96-3c9a614df517 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:40.532100 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:40.560470 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9409429f-ad69-4b3d-9825-225ad6fcab5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:40.561277 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409429f-ad69-4b3d-9825-225ad6fcab5b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:40.561557 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409429f-ad69-4b3d-9825-225ad6fcab5b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:40.561829 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9409429f-ad69-4b3d-9825-225ad6fcab5b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:40.569020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef471e7-5a6a-49ab-a709-0e28205ca5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:40.569905 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef471e7-5a6a-49ab-a709-0e28205ca5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:40.570330 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef471e7-5a6a-49ab-a709-0e28205ca5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:40.570749 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef471e7-5a6a-49ab-a709-0e28205ca5e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:40.571661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef471e7-5a6a-49ab-a709-0e28205ca5e2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:40.572135 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef471e7-5a6a-49ab-a709-0e28205ca5e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:40.573664 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef471e7-5a6a-49ab-a709-0e28205ca5e2 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:40.584875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 144/285] 10.222.0.24 () {56 vars in 1101 bytes} [Fri May 27 03:14:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:40.591779 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9409429f-ad69-4b3d-9825-225ad6fcab5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:40.600392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-9409429f-ad69-4b3d-9825-225ad6fcab5b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:40.600392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:40.600392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:40.600392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:40.600392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:40.600392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:40.600392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:40.600392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:40.600392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:40.600392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:40.600392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:40.600392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:40.600392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:40.600392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:40.600392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:40.600392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:40.601293 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:40.601293 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:40.601293 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:40.601293 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:40.601293 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:40.601293 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:40.601293 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:40.601293 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:40.601293 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:40.601293 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:40.601293 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:40.601293 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:14:40.601293 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:40.601293 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:40.601293 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:40.601293 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:40.602091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:40.602091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:40.602091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:40.602091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:40.602091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:40.602091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:40.602091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:40.602091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:40.602091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:40.602091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:40.602091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:40.602091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:14:40.602091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 142/286] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:40.603022 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef471e7-5a6a-49ab-a709-0e28205ca5e2 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:40.611904 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9409429f-ad69-4b3d-9825-225ad6fcab5b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:40.731389 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef471e7-5a6a-49ab-a709-0e28205ca5e2 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:40.783612 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b2b4d-c009-4354-9d29-d5f4f406dc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:40.784733 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b2b4d-c009-4354-9d29-d5f4f406dc46 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:40.786632 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b2b4d-c009-4354-9d29-d5f4f406dc46 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:40.787088 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b2b4d-c009-4354-9d29-d5f4f406dc46 None admin] PATH_INFO: `/v3/users/5ba57ca07c704b99bc61f73a3c129cdc` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:40.789417 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b2b4d-c009-4354-9d29-d5f4f406dc46 None admin] RBAC: Authorizing `identity:get_user(user_id=5ba57ca07c704b99bc61f73a3c129cdc)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:40.796005 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b2b4d-c009-4354-9d29-d5f4f406dc46 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:40.797872 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 145/287] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:40 2022] GET /identity/v3/users/5ba57ca07c704b99bc61f73a3c129cdc => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:40.803407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b2b4d-c009-4354-9d29-d5f4f406dc46 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:40.856617 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:40.876870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9409429f-ad69-4b3d-9825-225ad6fcab5b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:40.922096 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b2b4d-c009-4354-9d29-d5f4f406dc46 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:40.963735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211a29b-8d71-4050-a679-46cecf5faaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:40.964653 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211a29b-8d71-4050-a679-46cecf5faaf0 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:40.965082 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211a29b-8d71-4050-a679-46cecf5faaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:40.965641 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211a29b-8d71-4050-a679-46cecf5faaf0 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:40.966889 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3211a29b-8d71-4050-a679-46cecf5faaf0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:40.968321 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3211a29b-8d71-4050-a679-46cecf5faaf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:40.969858 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3211a29b-8d71-4050-a679-46cecf5faaf0 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:40.984738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871daa42-cbb9-4c1c-bd3c-476ed7d37fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:40.986222 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871daa42-cbb9-4c1c-bd3c-476ed7d37fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:40.986507 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871daa42-cbb9-4c1c-bd3c-476ed7d37fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:40.986787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871daa42-cbb9-4c1c-bd3c-476ed7d37fc0 None admin] PATH_INFO: `/v3/projects/d95eae40782d41759de176dea214eb16` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:40.989322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 143/288] 10.222.0.24 () {56 vars in 1101 bytes} [Fri May 27 03:14:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:40.990352 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871daa42-cbb9-4c1c-bd3c-476ed7d37fc0 None admin] RBAC: Authorizing `identity:get_project(project_id=d95eae40782d41759de176dea214eb16)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:40.992956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871daa42-cbb9-4c1c-bd3c-476ed7d37fc0 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:40.995084 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 146/289] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:40 2022] GET /identity/v3/projects/d95eae40782d41759de176dea214eb16 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:41.001230 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211a29b-8d71-4050-a679-46cecf5faaf0 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:41.003096 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871daa42-cbb9-4c1c-bd3c-476ed7d37fc0 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:41.067193 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:41.146797 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871daa42-cbb9-4c1c-bd3c-476ed7d37fc0 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:41.151251 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:41.191215 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211a29b-8d71-4050-a679-46cecf5faaf0 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:41.229166 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84c511-f4f2-4375-82dc-15c73bbe251c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:41.230062 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84c511-f4f2-4375-82dc-15c73bbe251c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:41.230349 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84c511-f4f2-4375-82dc-15c73bbe251c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:41.231702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84c511-f4f2-4375-82dc-15c73bbe251c None admin] PATH_INFO: `/v3/users/f25a34bc71ea41118a7a7546903e977e` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:41.232972 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d84c511-f4f2-4375-82dc-15c73bbe251c None admin] RBAC: Authorizing `identity:get_user(user_id=f25a34bc71ea41118a7a7546903e977e)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:41.234540 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d84c511-f4f2-4375-82dc-15c73bbe251c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:41.236762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 144/290] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:41 2022] GET /identity/v3/users/f25a34bc71ea41118a7a7546903e977e => generated 297 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:41.242325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84c511-f4f2-4375-82dc-15c73bbe251c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:41.257706 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4ef61-11df-4d9e-b756-86efeedfb8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:41.258666 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4ef61-11df-4d9e-b756-86efeedfb8a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:41.259014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4ef61-11df-4d9e-b756-86efeedfb8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:41.259291 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4ef61-11df-4d9e-b756-86efeedfb8a3 None admin] PATH_INFO: `/v3/projects/d95eae40782d41759de176dea214eb16/users/5ba57ca07c704b99bc61f73a3c129cdc/roles/aa0d3c6b36a24a718f88d0f4309d26d9` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:41.262874 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4ef61-11df-4d9e-b756-86efeedfb8a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=d95eae40782d41759de176dea214eb16, user_id=5ba57ca07c704b99bc61f73a3c129cdc, role_id=aa0d3c6b36a24a718f88d0f4309d26d9)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:41.264327 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4ef61-11df-4d9e-b756-86efeedfb8a3 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:41.360323 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 147/291] 10.222.0.24 () {58 vars in 1439 bytes} [Fri May 27 03:14:41 2022] PUT /identity/v3/projects/d95eae40782d41759de176dea214eb16/users/5ba57ca07c704b99bc61f73a3c129cdc/roles/aa0d3c6b36a24a718f88d0f4309d26d9 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:14:41.369835 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84c511-f4f2-4375-82dc-15c73bbe251c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:41.379164 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:41.408749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb081df-8a46-490a-b474-183f64f7348d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:41.409409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb081df-8a46-490a-b474-183f64f7348d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:41.409692 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb081df-8a46-490a-b474-183f64f7348d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:41.409967 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb081df-8a46-490a-b474-183f64f7348d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:41.483484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb081df-8a46-490a-b474-183f64f7348d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:41.485074 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb081df-8a46-490a-b474-183f64f7348d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:41.592566 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 145/292] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:14:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:14:41.598585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4ef61-11df-4d9e-b756-86efeedfb8a3 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:41.721968 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4ef61-11df-4d9e-b756-86efeedfb8a3 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:41.756242 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3cdcc-897c-4f6d-bdb8-0cdaa2b504f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:41.757105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3cdcc-897c-4f6d-bdb8-0cdaa2b504f0 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:41.757385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3cdcc-897c-4f6d-bdb8-0cdaa2b504f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:41.757655 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3cdcc-897c-4f6d-bdb8-0cdaa2b504f0 None admin] PATH_INFO: `/v3/projects/d95eae40782d41759de176dea214eb16` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:41.758828 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3cdcc-897c-4f6d-bdb8-0cdaa2b504f0 None admin] RBAC: Authorizing `identity:get_project(project_id=d95eae40782d41759de176dea214eb16)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:41.760339 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3cdcc-897c-4f6d-bdb8-0cdaa2b504f0 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:41.762900 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 148/293] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:41 2022] GET /identity/v3/projects/d95eae40782d41759de176dea214eb16 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:41.768368 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb081df-8a46-490a-b474-183f64f7348d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:41.777255 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:41.879950 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb081df-8a46-490a-b474-183f64f7348d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:41.914465 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a6d2a7-5528-47b0-af35-a58604e5f556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:41.915148 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6d2a7-5528-47b0-af35-a58604e5f556 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:41.915426 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6d2a7-5528-47b0-af35-a58604e5f556 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:41.915695 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a6d2a7-5528-47b0-af35-a58604e5f556 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:41.916594 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a6d2a7-5528-47b0-af35-a58604e5f556 None admin] RBAC: Adding query filter params (role.id=343337c6f25943a9ad412047b70ddefa, scope.project.id=d95eae40782d41759de176dea214eb16, user.id=f25a34bc71ea41118a7a7546903e977e) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:41.916902 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a6d2a7-5528-47b0-af35-a58604e5f556 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:41.918434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a6d2a7-5528-47b0-af35-a58604e5f556 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:41.924367 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 146/294] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:14:41 2022] GET /identity/v3/role_assignments?user.id=f25a34bc71ea41118a7a7546903e977e&scope.project.id=d95eae40782d41759de176dea214eb16&role.id=343337c6f25943a9ad412047b70ddefa => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:42.526949 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bf0dd-8ab9-42bf-8fc5-e1c927abaca2 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:42.527423 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bf0dd-8ab9-42bf-8fc5-e1c927abaca2 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:42.527423 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bf0dd-8ab9-42bf-8fc5-e1c927abaca2 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:42.529577 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 149/295] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:42.534951 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b5b93-022c-4b4f-829e-0f1efff40f3c None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:42.535245 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b5b93-022c-4b4f-829e-0f1efff40f3c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:42.535507 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b5b93-022c-4b4f-829e-0f1efff40f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:42.555482 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-1d3b5b93-022c-4b4f-829e-0f1efff40f3c None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:42.556668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1d3b5b93-022c-4b4f-829e-0f1efff40f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:42.630904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395dd887-94a4-4293-99b4-245ec5f1c0b1 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:42.631210 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395dd887-94a4-4293-99b4-245ec5f1c0b1 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:42.631456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395dd887-94a4-4293-99b4-245ec5f1c0b1 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:42.633048 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 150/296] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:14:42.633345 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:42.637937 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 147/297] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:42.644896 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3faf5-04f7-4409-adca-f3397f668dfc None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:42.644896 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3faf5-04f7-4409-adca-f3397f668dfc None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:42.644896 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3faf5-04f7-4409-adca-f3397f668dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:42.660889 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-57a3faf5-04f7-4409-adca-f3397f668dfc None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:42.662919 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b5b93-022c-4b4f-829e-0f1efff40f3c None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:42.663493 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-57a3faf5-04f7-4409-adca-f3397f668dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:42.682671 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:42.741149 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 151/298] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:42.766274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3faf5-04f7-4409-adca-f3397f668dfc None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:42.793212 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b5b93-022c-4b4f-829e-0f1efff40f3c None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:42.838687 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f27b0-5ad9-4514-b838-08111a794d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:42.841066 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f27b0-5ad9-4514-b838-08111a794d18 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:42.841527 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f27b0-5ad9-4514-b838-08111a794d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:42.841960 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f27b0-5ad9-4514-b838-08111a794d18 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:42.855105 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f27b0-5ad9-4514-b838-08111a794d18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:42.865272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-ef9f27b0-5ad9-4514-b838-08111a794d18 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:14:42.865272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:42.865272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:42.865272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:42.865272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:42.865272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:42.865272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:42.865272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:42.865272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:42.865272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:42.865272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:42.865272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:42.865272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:42.865272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:42.865272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:42.865272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:42.866356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:42.866356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:42.866356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:42.866356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:42.866356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:42.866356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:42.866356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:42.866356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:42.866356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:42.866356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:42.866356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:42.866356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:14:42.866356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:42.866356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:42.866356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:42.866356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:42.867360 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:42.867360 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:42.867360 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:42.867360 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:42.867360 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:42.867360 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:42.867360 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:42.867360 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:42.867360 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:42.867360 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:42.867360 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:14:42.867360 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:14:42.870023 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 148/299] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:14:42.875238 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f27b0-5ad9-4514-b838-08111a794d18 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:42.878667 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3faf5-04f7-4409-adca-f3397f668dfc None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:42.916622 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13718a42-a644-4b17-97f4-57766733464c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:42.917424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13718a42-a644-4b17-97f4-57766733464c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:42.918134 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13718a42-a644-4b17-97f4-57766733464c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:42.919015 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13718a42-a644-4b17-97f4-57766733464c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:42.920298 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13718a42-a644-4b17-97f4-57766733464c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:42.922443 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13718a42-a644-4b17-97f4-57766733464c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:42.924560 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 152/300] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:14:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:42.925630 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d16d24-1887-4c21-be0a-4aca9b390737 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:42.925916 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d16d24-1887-4c21-be0a-4aca9b390737 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:42.926215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d16d24-1887-4c21-be0a-4aca9b390737 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:42.930975 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 153/301] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:42.932393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d16d24-1887-4c21-be0a-4aca9b390737 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:42.942642 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:42.956154 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:42.987976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f27b0-5ad9-4514-b838-08111a794d18 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:43.022453 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838bbf74-7970-4860-bc64-68ce71742d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:43.023597 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bbf74-7970-4860-bc64-68ce71742d6b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:43.024087 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bbf74-7970-4860-bc64-68ce71742d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:43.024561 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bbf74-7970-4860-bc64-68ce71742d6b None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:43.025499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838bbf74-7970-4860-bc64-68ce71742d6b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:43.026004 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838bbf74-7970-4860-bc64-68ce71742d6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:43.027653 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838bbf74-7970-4860-bc64-68ce71742d6b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:43.036165 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 149/302] 10.222.0.24 () {56 vars in 1101 bytes} [Fri May 27 03:14:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:43.036632 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d16d24-1887-4c21-be0a-4aca9b390737 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:43.037840 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72eb3e6-1c32-4757-befb-f8e9c530a294 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:43.038330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72eb3e6-1c32-4757-befb-f8e9c530a294 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:43.038824 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72eb3e6-1c32-4757-befb-f8e9c530a294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:43.055897 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d72eb3e6-1c32-4757-befb-f8e9c530a294 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:43.057264 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d72eb3e6-1c32-4757-befb-f8e9c530a294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:43.073108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208ade2-4a8e-4dbd-9b09-de710197765e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:43.073905 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208ade2-4a8e-4dbd-9b09-de710197765e None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:43.074336 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208ade2-4a8e-4dbd-9b09-de710197765e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:43.075491 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208ade2-4a8e-4dbd-9b09-de710197765e None admin] PATH_INFO: `/v3/groups` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:43.076934 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b208ade2-4a8e-4dbd-9b09-de710197765e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:43.078951 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b208ade2-4a8e-4dbd-9b09-de710197765e None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:43.125785 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 150/303] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:43.127527 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72eb3e6-1c32-4757-befb-f8e9c530a294 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:43.161443 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 154/304] 10.222.0.24 () {60 vars in 1134 bytes} [Fri May 27 03:14:42 2022] POST /identity/v3/groups => generated 229 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:14:43.162268 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208ade2-4a8e-4dbd-9b09-de710197765e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:43.192364 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:43.261768 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72eb3e6-1c32-4757-befb-f8e9c530a294 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:43.263204 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:43.286708 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208ade2-4a8e-4dbd-9b09-de710197765e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:43.314027 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa5437-01bf-431f-9412-93718340f9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:43.318965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa5437-01bf-431f-9412-93718340f9d9 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:43.318965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa5437-01bf-431f-9412-93718340f9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:43.318965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa5437-01bf-431f-9412-93718340f9d9 None admin] PATH_INFO: `/v3/users/af7b00cbaabc44b68c56e8a29c253295` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:43.341962 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fa5437-01bf-431f-9412-93718340f9d9 None admin] RBAC: Authorizing `identity:get_user(user_id=af7b00cbaabc44b68c56e8a29c253295)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:43.344349 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fa5437-01bf-431f-9412-93718340f9d9 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:43.346756 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 151/305] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:43 2022] GET /identity/v3/users/af7b00cbaabc44b68c56e8a29c253295 => generated 293 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:43.349245 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059f249-d00b-4541-88ad-47b6ca95bb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:43.350054 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059f249-d00b-4541-88ad-47b6ca95bb0b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:43.350337 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059f249-d00b-4541-88ad-47b6ca95bb0b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:43.350611 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059f249-d00b-4541-88ad-47b6ca95bb0b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:43.354865 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa5437-01bf-431f-9412-93718340f9d9 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:43.366083 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c059f249-d00b-4541-88ad-47b6ca95bb0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:43.375927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-c059f249-d00b-4541-88ad-47b6ca95bb0b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:43.375927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:43.375927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:43.375927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:43.375927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:43.375927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:43.375927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:43.375927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:43.375927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:43.375927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:43.375927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:43.375927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:43.375927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:43.375927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:43.375927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:43.375927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:43.376827 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:43.376827 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:43.376827 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:43.376827 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:43.376827 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:43.376827 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:43.376827 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:43.376827 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:43.376827 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:43.376827 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:43.376827 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:43.376827 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:14:43.376827 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:43.376827 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:43.376827 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:43.376827 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:43.383521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:43.383521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:43.383521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:43.383521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:43.383521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:43.383521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:43.383521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:43.383521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:43.383521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:43.383521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:43.383521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:43.383521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:14:43.383521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 155/306] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:43.383521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059f249-d00b-4541-88ad-47b6ca95bb0b None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:43.481588 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa5437-01bf-431f-9412-93718340f9d9 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:43.493939 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:43.513292 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059f249-d00b-4541-88ad-47b6ca95bb0b None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:43.521601 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee197a-a73c-4ea6-82a9-4bbab1e46dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:43.522663 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee197a-a73c-4ea6-82a9-4bbab1e46dec None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:43.523165 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee197a-a73c-4ea6-82a9-4bbab1e46dec None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:43.523712 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ee197a-a73c-4ea6-82a9-4bbab1e46dec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:43.524546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee197a-a73c-4ea6-82a9-4bbab1e46dec None admin] RBAC: Adding query filter params (role.id=aa0d3c6b36a24a718f88d0f4309d26d9, user.id=af7b00cbaabc44b68c56e8a29c253295, scope.system=all) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:43.525070 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee197a-a73c-4ea6-82a9-4bbab1e46dec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:43.526281 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ee197a-a73c-4ea6-82a9-4bbab1e46dec None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:43.532459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 152/307] 10.222.0.24 () {56 vars in 1308 bytes} [Fri May 27 03:14:43 2022] GET /identity/v3/role_assignments?user.id=af7b00cbaabc44b68c56e8a29c253295&scope.system=all&role.id=aa0d3c6b36a24a718f88d0f4309d26d9 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:43.556359 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be895fc9-d331-4e2c-8a77-08b0b0ebcb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:43.557750 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be895fc9-d331-4e2c-8a77-08b0b0ebcb88 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:43.558236 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be895fc9-d331-4e2c-8a77-08b0b0ebcb88 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:43.558709 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be895fc9-d331-4e2c-8a77-08b0b0ebcb88 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:43.560274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be895fc9-d331-4e2c-8a77-08b0b0ebcb88 None admin] RBAC: Adding query filter params (name=member) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:43.560822 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be895fc9-d331-4e2c-8a77-08b0b0ebcb88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:43.562496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be895fc9-d331-4e2c-8a77-08b0b0ebcb88 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:43.573270 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 156/308] 10.222.0.24 () {56 vars in 1101 bytes} [Fri May 27 03:14:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:43.579679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee197a-a73c-4ea6-82a9-4bbab1e46dec None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:43.695562 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ee197a-a73c-4ea6-82a9-4bbab1e46dec None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:43.741459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ce7d6-f6bd-4043-8c6c-1127ccf9a138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:43.742698 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ce7d6-f6bd-4043-8c6c-1127ccf9a138 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:43.743213 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ce7d6-f6bd-4043-8c6c-1127ccf9a138 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:43.743689 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ce7d6-f6bd-4043-8c6c-1127ccf9a138 None admin] PATH_INFO: `/v3/users/21c88ba21e1141efafca737507619cb9` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:43.745093 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ce7d6-f6bd-4043-8c6c-1127ccf9a138 None admin] RBAC: Authorizing `identity:get_user(user_id=21c88ba21e1141efafca737507619cb9)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:43.746828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ce7d6-f6bd-4043-8c6c-1127ccf9a138 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:43.748998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 153/309] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:43 2022] GET /identity/v3/users/21c88ba21e1141efafca737507619cb9 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:43.754669 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be895fc9-d331-4e2c-8a77-08b0b0ebcb88 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:43.757949 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:43.866404 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be895fc9-d331-4e2c-8a77-08b0b0ebcb88 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:43.904956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8200e29e-9c18-4761-af03-880542a79a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:43.905826 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200e29e-9c18-4761-af03-880542a79a61 None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:43.906107 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200e29e-9c18-4761-af03-880542a79a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:43.906384 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200e29e-9c18-4761-af03-880542a79a61 None admin] PATH_INFO: `/v3/system/users/21c88ba21e1141efafca737507619cb9/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:43.907916 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8200e29e-9c18-4761-af03-880542a79a61 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=21c88ba21e1141efafca737507619cb9, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:43.909291 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8200e29e-9c18-4761-af03-880542a79a61 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:43.974672 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cc332-0443-4fee-ab60-f909f2c8b446 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:43.975207 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cc332-0443-4fee-ab60-f909f2c8b446 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:43.975282 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cc332-0443-4fee-ab60-f909f2c8b446 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:43.976913 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 154/310] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:43.981882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99767674-5e1a-4b34-8803-3979a18d7162 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:43.982171 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99767674-5e1a-4b34-8803-3979a18d7162 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:43.982458 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99767674-5e1a-4b34-8803-3979a18d7162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:44.000882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-99767674-5e1a-4b34-8803-3979a18d7162 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:44.001956 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-99767674-5e1a-4b34-8803-3979a18d7162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:44.003321 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 157/311] 10.222.0.24 () {58 vars in 1334 bytes} [Fri May 27 03:14:43 2022] PUT /identity/v3/system/users/21c88ba21e1141efafca737507619cb9/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:14:44.052897 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:44.103806 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 155/312] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:14:44.128724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8200e29e-9c18-4761-af03-880542a79a61 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:44.202284 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:44.243900 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8200e29e-9c18-4761-af03-880542a79a61 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:44.280291 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dded1-cad9-49b6-b519-eac50bff1f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:44.281462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dded1-cad9-49b6-b519-eac50bff1f47 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:44.281947 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dded1-cad9-49b6-b519-eac50bff1f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:44.282421 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80dded1-cad9-49b6-b519-eac50bff1f47 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:44.293524 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80dded1-cad9-49b6-b519-eac50bff1f47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:44.301687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-a80dded1-cad9-49b6-b519-eac50bff1f47 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:14:44.301687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:44.301687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:44.301687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:44.301687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:44.301687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:44.301687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:44.301687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:44.301687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:44.301687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:44.301687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:44.301687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:44.301687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:44.301687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:44.301687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:44.301687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:44.302562 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:44.302562 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:44.302562 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:44.302562 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:44.302562 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:44.302562 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:44.302562 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:44.302562 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:44.302562 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:44.302562 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:44.302562 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:44.302562 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:14:44.302562 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:44.302562 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:44.302562 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:44.302562 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:44.303380 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:44.303380 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:44.303380 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:44.303380 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:44.303380 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:44.303380 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:44.303380 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:44.303380 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:44.303380 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:44.303380 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:44.303380 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:14:44.303380 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:14:44.306000 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 158/313] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:44.312474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99767674-5e1a-4b34-8803-3979a18d7162 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:44.416764 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99767674-5e1a-4b34-8803-3979a18d7162 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:44.451987 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60186edf-c754-47df-968f-1f1e230aed38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:44.452659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60186edf-c754-47df-968f-1f1e230aed38 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:44.452936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60186edf-c754-47df-968f-1f1e230aed38 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:44.453206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60186edf-c754-47df-968f-1f1e230aed38 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:44.453954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60186edf-c754-47df-968f-1f1e230aed38 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:44.454260 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60186edf-c754-47df-968f-1f1e230aed38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:44.455667 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60186edf-c754-47df-968f-1f1e230aed38 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:44.464321 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 156/314] 10.222.0.24 () {56 vars in 1101 bytes} [Fri May 27 03:14:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:44.469723 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dded1-cad9-49b6-b519-eac50bff1f47 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:44.577808 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80dded1-cad9-49b6-b519-eac50bff1f47 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:44.603781 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:44.612696 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d872c9-955a-4f2e-8bf1-8ccd423930ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:44.613576 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d872c9-955a-4f2e-8bf1-8ccd423930ea None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:44.613905 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d872c9-955a-4f2e-8bf1-8ccd423930ea None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:44.614182 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d872c9-955a-4f2e-8bf1-8ccd423930ea None admin] PATH_INFO: `/v3/users/5ba57ca07c704b99bc61f73a3c129cdc` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:44.627919 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d872c9-955a-4f2e-8bf1-8ccd423930ea None admin] RBAC: Authorizing `identity:get_user(user_id=5ba57ca07c704b99bc61f73a3c129cdc)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:44.627919 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d872c9-955a-4f2e-8bf1-8ccd423930ea None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:44.627919 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 159/315] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:44 2022] GET /identity/v3/users/5ba57ca07c704b99bc61f73a3c129cdc => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:44.628144 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60186edf-c754-47df-968f-1f1e230aed38 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:44.662942 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:44.739662 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60186edf-c754-47df-968f-1f1e230aed38 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:44.775466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbded8-d2d3-4a26-9796-ab5dd13900d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:44.776113 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbded8-d2d3-4a26-9796-ab5dd13900d8 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:44.776396 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbded8-d2d3-4a26-9796-ab5dd13900d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:44.776672 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbded8-d2d3-4a26-9796-ab5dd13900d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:44.845593 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fbded8-d2d3-4a26-9796-ab5dd13900d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:44.847082 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fbded8-d2d3-4a26-9796-ab5dd13900d8 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:44.887561 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:44.947248 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 157/316] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:14:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:14:44.952687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d872c9-955a-4f2e-8bf1-8ccd423930ea None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:45.115691 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d872c9-955a-4f2e-8bf1-8ccd423930ea None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:45.150545 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dcdc70-987d-4bde-8ff6-f6aa1468ed9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:45.151448 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dcdc70-987d-4bde-8ff6-f6aa1468ed9a None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:45.151729 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dcdc70-987d-4bde-8ff6-f6aa1468ed9a None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:45.152002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dcdc70-987d-4bde-8ff6-f6aa1468ed9a None admin] PATH_INFO: `/v3/projects/d95eae40782d41759de176dea214eb16` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:45.153344 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dcdc70-987d-4bde-8ff6-f6aa1468ed9a None admin] RBAC: Authorizing `identity:get_project(project_id=d95eae40782d41759de176dea214eb16)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:45.154954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dcdc70-987d-4bde-8ff6-f6aa1468ed9a None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:45.157039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 160/317] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:44 2022] GET /identity/v3/projects/d95eae40782d41759de176dea214eb16 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:45.162764 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbded8-d2d3-4a26-9796-ab5dd13900d8 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:45.267901 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbded8-d2d3-4a26-9796-ab5dd13900d8 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:45.302642 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8cd36-f4e6-446c-90d2-b39b7cf861b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:45.303327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8cd36-f4e6-446c-90d2-b39b7cf861b1 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:45.303607 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8cd36-f4e6-446c-90d2-b39b7cf861b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:45.303878 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8cd36-f4e6-446c-90d2-b39b7cf861b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:45.304709 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8cd36-f4e6-446c-90d2-b39b7cf861b1 None admin] RBAC: Adding query filter params (role.id=aa0d3c6b36a24a718f88d0f4309d26d9, scope.project.id=d95eae40782d41759de176dea214eb16, user.id=5ba57ca07c704b99bc61f73a3c129cdc) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:45.305020 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8cd36-f4e6-446c-90d2-b39b7cf861b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:45.306461 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8cd36-f4e6-446c-90d2-b39b7cf861b1 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:45.308225 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:45.313018 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 158/318] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:14:45 2022] GET /identity/v3/role_assignments?user.id=5ba57ca07c704b99bc61f73a3c129cdc&scope.project.id=d95eae40782d41759de176dea214eb16&role.id=aa0d3c6b36a24a718f88d0f4309d26d9 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:45.562170 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866e40c-5af7-4405-b624-50eceef01139 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:45.562902 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866e40c-5af7-4405-b624-50eceef01139 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:45.563404 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866e40c-5af7-4405-b624-50eceef01139 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:45.564955 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 161/319] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:45.570446 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7790360-c700-4673-a239-2ae5c01682b6 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:45.570737 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7790360-c700-4673-a239-2ae5c01682b6 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:45.571046 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7790360-c700-4673-a239-2ae5c01682b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:45.588912 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d7790360-c700-4673-a239-2ae5c01682b6 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:45.590703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d7790360-c700-4673-a239-2ae5c01682b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:45.663900 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 159/320] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:45.686925 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7866e40c-5af7-4405-b624-50eceef01139 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:45.750658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:45.791262 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7866e40c-5af7-4405-b624-50eceef01139 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:45.824140 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793922a-2668-48b8-b534-c5b128f1d407 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:45.824379 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793922a-2668-48b8-b534-c5b128f1d407 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:45.824684 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793922a-2668-48b8-b534-c5b128f1d407 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:45.825175 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06829330-ccab-4977-a87a-47f6b2bcb6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:45.825972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06829330-ccab-4977-a87a-47f6b2bcb6be None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:45.826555 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 160/321] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:14:45.826651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06829330-ccab-4977-a87a-47f6b2bcb6be None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:45.826885 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06829330-ccab-4977-a87a-47f6b2bcb6be None admin] PATH_INFO: `/v3/domains/default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:45.828092 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06829330-ccab-4977-a87a-47f6b2bcb6be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:45.829536 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06829330-ccab-4977-a87a-47f6b2bcb6be None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:45.835799 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 162/322] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:14:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:45.835930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc6ae8-a03b-4766-a2d4-311ee8b84983 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:45.835930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc6ae8-a03b-4766-a2d4-311ee8b84983 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:45.835930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc6ae8-a03b-4766-a2d4-311ee8b84983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:45.836508 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06829330-ccab-4977-a87a-47f6b2bcb6be None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:45.851183 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-86bc6ae8-a03b-4766-a2d4-311ee8b84983 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:45.852367 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-86bc6ae8-a03b-4766-a2d4-311ee8b84983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:45.921610 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 161/323] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:14:45.942878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06829330-ccab-4977-a87a-47f6b2bcb6be None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:45.943446 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc6ae8-a03b-4766-a2d4-311ee8b84983 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:45.973394 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:45.979194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b35ee-11f4-47bb-9b9d-ca1407e4d7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:45.979861 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b35ee-11f4-47bb-9b9d-ca1407e4d7ca None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:45.980145 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b35ee-11f4-47bb-9b9d-ca1407e4d7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:45.980423 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b35ee-11f4-47bb-9b9d-ca1407e4d7ca None admin] PATH_INFO: `/v3/groups` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:45.981750 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6b35ee-11f4-47bb-9b9d-ca1407e4d7ca None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:45.983113 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6b35ee-11f4-47bb-9b9d-ca1407e4d7ca None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:46.047573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:46.086412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 163/324] 10.222.0.24 () {60 vars in 1134 bytes} [Fri May 27 03:14:45 2022] POST /identity/v3/groups => generated 226 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:14:46.099994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc6ae8-a03b-4766-a2d4-311ee8b84983 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:46.141178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d80b9-c611-44bc-811b-5aa4f54de334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:46.142001 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d80b9-c611-44bc-811b-5aa4f54de334 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:46.142283 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d80b9-c611-44bc-811b-5aa4f54de334 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:46.142556 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d80b9-c611-44bc-811b-5aa4f54de334 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:46.154543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d80b9-c611-44bc-811b-5aa4f54de334 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:46.162512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-230d80b9-c611-44bc-811b-5aa4f54de334 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:14:46.162512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:46.162512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:46.162512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:46.162512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:46.162512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:46.162512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:46.162512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:46.162512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:46.162512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:46.162512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:46.162512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:46.162512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:46.162512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:46.162512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:46.162512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:46.163950 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:46.163950 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:46.163950 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:46.163950 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:46.163950 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:46.163950 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:46.163950 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:46.163950 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:46.163950 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:46.163950 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:46.163950 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:46.163950 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:14:46.163950 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:46.163950 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:46.163950 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:46.163950 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:46.164743 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:46.164743 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:46.164743 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:46.164743 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:46.164743 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:46.164743 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:46.164743 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:46.164743 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:46.164743 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:46.164743 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:46.164743 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:14:46.164743 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:14:46.167202 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 162/325] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:14:46.170363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b35ee-11f4-47bb-9b9d-ca1407e4d7ca None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:46.281439 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa34f76-b2b4-488f-8597-7e3550eef6f5 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:46.281751 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa34f76-b2b4-488f-8597-7e3550eef6f5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:46.281850 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b35ee-11f4-47bb-9b9d-ca1407e4d7ca None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:46.282023 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa34f76-b2b4-488f-8597-7e3550eef6f5 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:46.283647 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 163/326] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:14:46.288677 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3712498-4ea6-44b8-ae0c-3926bd95cbcf None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:46.289244 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3712498-4ea6-44b8-ae0c-3926bd95cbcf None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:46.290072 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3712498-4ea6-44b8-ae0c-3926bd95cbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:46.310118 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-b3712498-4ea6-44b8-ae0c-3926bd95cbcf None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:46.311492 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b3712498-4ea6-44b8-ae0c-3926bd95cbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:46.317799 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9f5e3-f6f0-4b27-9ca3-b4861d0880a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:46.318476 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9f5e3-f6f0-4b27-9ca3-b4861d0880a0 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:46.318759 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9f5e3-f6f0-4b27-9ca3-b4861d0880a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:46.319060 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9f5e3-f6f0-4b27-9ca3-b4861d0880a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:46.319811 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9f5e3-f6f0-4b27-9ca3-b4861d0880a0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:46.320119 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9f5e3-f6f0-4b27-9ca3-b4861d0880a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:46.321490 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9f5e3-f6f0-4b27-9ca3-b4861d0880a0 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:46.331643 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 164/327] 10.222.0.24 () {56 vars in 1101 bytes} [Fri May 27 03:14:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:46.337984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9f5e3-f6f0-4b27-9ca3-b4861d0880a0 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:46.397715 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 164/328] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:14:46.422656 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3712498-4ea6-44b8-ae0c-3926bd95cbcf None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:46.423260 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:46.460854 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9f5e3-f6f0-4b27-9ca3-b4861d0880a0 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:46.492464 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:46.507233 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e0d2c-02f3-406a-af19-a6f54f5e7b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:46.508124 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e0d2c-02f3-406a-af19-a6f54f5e7b17 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:46.508405 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e0d2c-02f3-406a-af19-a6f54f5e7b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:46.508687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e0d2c-02f3-406a-af19-a6f54f5e7b17 None admin] PATH_INFO: `/v3/users/af7b00cbaabc44b68c56e8a29c253295` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:46.509901 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e0d2c-02f3-406a-af19-a6f54f5e7b17 None admin] RBAC: Authorizing `identity:get_user(user_id=af7b00cbaabc44b68c56e8a29c253295)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:46.511398 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e0d2c-02f3-406a-af19-a6f54f5e7b17 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:46.513945 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 165/329] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:46 2022] GET /identity/v3/users/af7b00cbaabc44b68c56e8a29c253295 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:46.519772 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e0d2c-02f3-406a-af19-a6f54f5e7b17 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:46.558618 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3712498-4ea6-44b8-ae0c-3926bd95cbcf None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:46.604998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e609206-0b62-4b2f-bcb4-89c406600bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:46.605786 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e609206-0b62-4b2f-bcb4-89c406600bfd None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:46.606070 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e609206-0b62-4b2f-bcb4-89c406600bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:46.606346 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e609206-0b62-4b2f-bcb4-89c406600bfd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:46.622504 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e609206-0b62-4b2f-bcb4-89c406600bfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:46.632744 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-8e609206-0b62-4b2f-bcb4-89c406600bfd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:46.632744 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:46.632744 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:46.632744 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:46.632744 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:46.632744 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:46.632744 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:46.632744 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:46.632744 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:46.632744 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:46.632744 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:46.632744 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:46.632744 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:46.632744 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:46.632744 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:46.632744 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:46.633630 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:46.633630 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:46.633630 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:46.633630 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:46.633630 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:46.633630 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:46.633630 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:46.633630 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:46.633630 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:46.633630 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:46.633630 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:46.633630 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:14:46.633630 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:46.633630 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:46.633630 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:46.633630 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:46.634421 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:46.634421 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:46.634421 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:46.634421 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:46.634421 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:46.634421 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:46.634421 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:46.634421 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:46.634421 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:46.634421 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:46.634421 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:46.634421 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:14:46.636696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 165/330] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:14:46.643245 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e609206-0b62-4b2f-bcb4-89c406600bfd None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:46.661111 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e0d2c-02f3-406a-af19-a6f54f5e7b17 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:46.689325 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:46.710388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0cc6f-3604-4d02-ac00-86f262826da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:46.711787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0cc6f-3604-4d02-ac00-86f262826da1 None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:46.712234 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0cc6f-3604-4d02-ac00-86f262826da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:46.712931 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0cc6f-3604-4d02-ac00-86f262826da1 None admin] PATH_INFO: `/v3/system/users/af7b00cbaabc44b68c56e8a29c253295/roles/aa0d3c6b36a24a718f88d0f4309d26d9` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:46.715356 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a0cc6f-3604-4d02-ac00-86f262826da1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=af7b00cbaabc44b68c56e8a29c253295, role_id=aa0d3c6b36a24a718f88d0f4309d26d9)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:46.717631 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a0cc6f-3604-4d02-ac00-86f262826da1 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:46.769213 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e609206-0b62-4b2f-bcb4-89c406600bfd None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:46.779513 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:46.785376 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 166/331] 10.222.0.24 () {58 vars in 1334 bytes} [Fri May 27 03:14:46 2022] PUT /identity/v3/system/users/af7b00cbaabc44b68c56e8a29c253295/roles/aa0d3c6b36a24a718f88d0f4309d26d9 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:14:46.806980 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00d9f1-69d3-452c-8355-2d36ed1cbce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:46.807734 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00d9f1-69d3-452c-8355-2d36ed1cbce6 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:46.808018 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00d9f1-69d3-452c-8355-2d36ed1cbce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:46.808295 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00d9f1-69d3-452c-8355-2d36ed1cbce6 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:46.809274 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00d9f1-69d3-452c-8355-2d36ed1cbce6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:46.810048 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00d9f1-69d3-452c-8355-2d36ed1cbce6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:46.811578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00d9f1-69d3-452c-8355-2d36ed1cbce6 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:46.821924 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 166/332] 10.222.0.24 () {56 vars in 1101 bytes} [Fri May 27 03:14:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:46.829030 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0cc6f-3604-4d02-ac00-86f262826da1 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:46.962372 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0cc6f-3604-4d02-ac00-86f262826da1 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:47.004039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde4c0d-62d9-4e79-bc7f-fe96f18261bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:47.004929 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde4c0d-62d9-4e79-bc7f-fe96f18261bf None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:47.005207 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde4c0d-62d9-4e79-bc7f-fe96f18261bf None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:47.005479 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde4c0d-62d9-4e79-bc7f-fe96f18261bf None admin] PATH_INFO: `/v3/users/21c88ba21e1141efafca737507619cb9` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:47.010248 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bde4c0d-62d9-4e79-bc7f-fe96f18261bf None admin] RBAC: Authorizing `identity:get_user(user_id=21c88ba21e1141efafca737507619cb9)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:47.011868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bde4c0d-62d9-4e79-bc7f-fe96f18261bf None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:47.014450 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 167/333] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:46 2022] GET /identity/v3/users/21c88ba21e1141efafca737507619cb9 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:47.020216 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00d9f1-69d3-452c-8355-2d36ed1cbce6 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:47.145387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00d9f1-69d3-452c-8355-2d36ed1cbce6 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:47.183117 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca3686-dd43-46a5-b459-05b3d2424a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:47.184392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca3686-dd43-46a5-b459-05b3d2424a4b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:47.184599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca3686-dd43-46a5-b459-05b3d2424a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:47.184874 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca3686-dd43-46a5-b459-05b3d2424a4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:47.185713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca3686-dd43-46a5-b459-05b3d2424a4b None admin] RBAC: Adding query filter params (role.id=343337c6f25943a9ad412047b70ddefa, user.id=21c88ba21e1141efafca737507619cb9, scope.system=all) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:47.186077 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca3686-dd43-46a5-b459-05b3d2424a4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:47.187595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca3686-dd43-46a5-b459-05b3d2424a4b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:47.200928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 167/334] 10.222.0.24 () {56 vars in 1308 bytes} [Fri May 27 03:14:47 2022] GET /identity/v3/role_assignments?user.id=21c88ba21e1141efafca737507619cb9&scope.system=all&role.id=343337c6f25943a9ad412047b70ddefa => generated 510 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:48.827533 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0794ac5-5e30-47e8-bca9-f94b47ec7aa3 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:48.828150 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0794ac5-5e30-47e8-bca9-f94b47ec7aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:48.828150 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0794ac5-5e30-47e8-bca9-f94b47ec7aa3 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:48.829670 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 168/335] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:48.835457 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f5142-3efa-4a07-b02b-04aa5e894678 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:48.836329 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f5142-3efa-4a07-b02b-04aa5e894678 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:48.836808 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f5142-3efa-4a07-b02b-04aa5e894678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:48.859903 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d51f5142-3efa-4a07-b02b-04aa5e894678 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:48.861448 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d51f5142-3efa-4a07-b02b-04aa5e894678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:48.889439 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:48.936098 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 168/336] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:14:48.984452 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0794ac5-5e30-47e8-bca9-f94b47ec7aa3 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:49.047447 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:49.128544 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0794ac5-5e30-47e8-bca9-f94b47ec7aa3 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:49.168123 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0315249a-4540-4c0a-accf-edebbbbf65fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:49.169204 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315249a-4540-4c0a-accf-edebbbbf65fd None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:49.169642 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315249a-4540-4c0a-accf-edebbbbf65fd None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:49.170071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315249a-4540-4c0a-accf-edebbbbf65fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:49.184381 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0315249a-4540-4c0a-accf-edebbbbf65fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:49.192438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-0315249a-4540-4c0a-accf-edebbbbf65fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:49.192438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:49.192438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:49.192438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:49.192438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:49.192438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:49.192438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:49.192438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:49.192438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:49.192438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:49.192438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:49.192438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:49.192438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:49.192438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:49.192438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:49.192438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:49.193563 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:49.193563 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:49.193563 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:49.193563 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:49.193563 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:49.193563 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:49.193563 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:49.193563 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:49.193563 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:49.193563 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:49.193563 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:49.193563 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:14:49.193563 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:49.193563 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:49.193563 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:49.193563 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:49.194560 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:49.194560 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:49.194560 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:49.194560 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:49.194560 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:49.194560 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:49.194560 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:49.194560 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:49.194560 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:49.194560 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:49.194560 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:49.194560 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:14:49.201807 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 169/337] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:49.203137 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f5142-3efa-4a07-b02b-04aa5e894678 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:49.374292 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f5142-3efa-4a07-b02b-04aa5e894678 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:49.425634 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08634c8e-8088-496b-bb4b-9ae6e50455f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:49.426330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08634c8e-8088-496b-bb4b-9ae6e50455f3 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:49.426611 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08634c8e-8088-496b-bb4b-9ae6e50455f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:49.426910 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08634c8e-8088-496b-bb4b-9ae6e50455f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:49.427695 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08634c8e-8088-496b-bb4b-9ae6e50455f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:49.428001 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08634c8e-8088-496b-bb4b-9ae6e50455f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:49.429421 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08634c8e-8088-496b-bb4b-9ae6e50455f3 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:49.441277 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 169/338] 10.222.0.24 () {56 vars in 1101 bytes} [Fri May 27 03:14:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:49.446579 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09562f12-63a5-4d36-897f-fc57b4ac72c0 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:49.446717 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08634c8e-8088-496b-bb4b-9ae6e50455f3 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:49.447256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09562f12-63a5-4d36-897f-fc57b4ac72c0 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:49.447851 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09562f12-63a5-4d36-897f-fc57b4ac72c0 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:49.449620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 170/339] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:49.457275 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e475119-c7a5-41ec-a517-06adfaae180f None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:49.457824 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e475119-c7a5-41ec-a517-06adfaae180f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:49.458318 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e475119-c7a5-41ec-a517-06adfaae180f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:49.473980 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:49.487643 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-1e475119-c7a5-41ec-a517-06adfaae180f None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:49.488898 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1e475119-c7a5-41ec-a517-06adfaae180f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:49.595092 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 171/340] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:14:49.634502 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08634c8e-8088-496b-bb4b-9ae6e50455f3 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:49.639084 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e475119-c7a5-41ec-a517-06adfaae180f None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:49.696644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283a216-21b4-4616-8fe0-ec5eb8c5b7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:49.697657 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283a216-21b4-4616-8fe0-ec5eb8c5b7ff None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:49.698094 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283a216-21b4-4616-8fe0-ec5eb8c5b7ff None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:49.698503 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283a216-21b4-4616-8fe0-ec5eb8c5b7ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:49.717689 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:49.823795 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e475119-c7a5-41ec-a517-06adfaae180f None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:49.855260 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6283a216-21b4-4616-8fe0-ec5eb8c5b7ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:49.856842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6283a216-21b4-4616-8fe0-ec5eb8c5b7ff None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:49.869997 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abbf459-5efd-40e9-a7a3-a4ba26867691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:49.870819 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbf459-5efd-40e9-a7a3-a4ba26867691 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:49.871160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbf459-5efd-40e9-a7a3-a4ba26867691 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:49.871433 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abbf459-5efd-40e9-a7a3-a4ba26867691 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:49.884734 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abbf459-5efd-40e9-a7a3-a4ba26867691 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:49.891199 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:49.894415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-3abbf459-5efd-40e9-a7a3-a4ba26867691 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:49.894415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:49.894415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:49.894415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:49.894415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:49.894415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:49.894415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:49.894415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:49.894415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:49.894415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:49.894415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:49.894415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:49.894415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:49.894415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:49.894415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:49.894415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:49.895383 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:49.895383 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:49.895383 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:49.895383 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:49.895383 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:49.895383 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:49.895383 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:49.895383 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:49.895383 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:49.895383 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:49.895383 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:49.895383 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:14:49.895383 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:49.895383 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:49.895383 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:49.895383 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:49.896180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:49.896180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:49.896180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:49.896180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:49.896180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:49.896180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:49.896180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:49.896180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:49.896180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:49.896180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:49.896180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:49.896180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:14:49.896180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 172/341] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:14:49.897625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a26e4-fb44-4058-8b69-eeca08b576b7 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:49.897736 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a26e4-fb44-4058-8b69-eeca08b576b7 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:49.899004 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a26e4-fb44-4058-8b69-eeca08b576b7 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:49.905815 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 173/342] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:49.905815 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a26e4-fb44-4058-8b69-eeca08b576b7 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:50.071823 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 170/343] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:14:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:14:50.075390 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a26e4-fb44-4058-8b69-eeca08b576b7 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:50.082860 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04226f3-2b23-42df-8f94-8c56b38823fd None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:50.082860 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04226f3-2b23-42df-8f94-8c56b38823fd None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:50.082860 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04226f3-2b23-42df-8f94-8c56b38823fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:50.086250 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:50.115933 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d04226f3-2b23-42df-8f94-8c56b38823fd None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:50.117136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d04226f3-2b23-42df-8f94-8c56b38823fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:50.145902 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0bcfaf-0719-43e4-9998-cf0bb4665c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:50.146506 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0bcfaf-0719-43e4-9998-cf0bb4665c3e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:50.146786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0bcfaf-0719-43e4-9998-cf0bb4665c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:50.147099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0bcfaf-0719-43e4-9998-cf0bb4665c3e None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:50.147860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0bcfaf-0719-43e4-9998-cf0bb4665c3e None admin] RBAC: Adding query filter params (name=member) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:50.148166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0bcfaf-0719-43e4-9998-cf0bb4665c3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:50.150050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0bcfaf-0719-43e4-9998-cf0bb4665c3e None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:50.161994 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 174/344] 10.222.0.24 () {56 vars in 1101 bytes} [Fri May 27 03:14:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:50.163244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0bcfaf-0719-43e4-9998-cf0bb4665c3e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:50.203937 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 171/345] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:50.205586 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47077360-518e-4300-8cd5-8dc727323a06 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:50.210858 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47077360-518e-4300-8cd5-8dc727323a06 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:50.211627 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47077360-518e-4300-8cd5-8dc727323a06 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:50.213622 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 172/346] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:50 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:50.214620 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47077360-518e-4300-8cd5-8dc727323a06 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:50.283608 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0bcfaf-0719-43e4-9998-cf0bb4665c3e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:50.301453 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:50.327468 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ab7dc-a45f-42cf-8e83-02fa384853c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:50.330664 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ab7dc-a45f-42cf-8e83-02fa384853c8 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:50.330664 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ab7dc-a45f-42cf-8e83-02fa384853c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:50.330664 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ab7dc-a45f-42cf-8e83-02fa384853c8 None admin] PATH_INFO: `/v3/projects/9dbc862abb4247f69fc1a65f462d65ce` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:50.330664 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3ab7dc-a45f-42cf-8e83-02fa384853c8 None admin] RBAC: Authorizing `identity:get_project(project_id=9dbc862abb4247f69fc1a65f462d65ce)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:50.331934 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3ab7dc-a45f-42cf-8e83-02fa384853c8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:50.334314 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 175/347] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:50 2022] GET /identity/v3/projects/9dbc862abb4247f69fc1a65f462d65ce => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:50.335870 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e75529-1ac6-41c3-bdc0-d77870a35a1a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:50.336161 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e75529-1ac6-41c3-bdc0-d77870a35a1a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:50.336714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e75529-1ac6-41c3-bdc0-d77870a35a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:50.357925 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-81e75529-1ac6-41c3-bdc0-d77870a35a1a None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:50.359151 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-81e75529-1ac6-41c3-bdc0-d77870a35a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:50.362154 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47077360-518e-4300-8cd5-8dc727323a06 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:50.392335 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:50.401806 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76fa47-cee5-4312-b14e-01f2de726a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:50.402586 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76fa47-cee5-4312-b14e-01f2de726a40 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:50.403066 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76fa47-cee5-4312-b14e-01f2de726a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:50.403390 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76fa47-cee5-4312-b14e-01f2de726a40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:50.439667 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 176/348] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:50.440759 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e75529-1ac6-41c3-bdc0-d77870a35a1a None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:50.496434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d76fa47-cee5-4312-b14e-01f2de726a40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:50.498189 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d76fa47-cee5-4312-b14e-01f2de726a40 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:50.547060 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e75529-1ac6-41c3-bdc0-d77870a35a1a None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:50.557572 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:50.585023 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e5e9a-7e4b-41b5-96a5-71349c75afda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:50.586240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e5e9a-7e4b-41b5-96a5-71349c75afda None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:50.586737 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e5e9a-7e4b-41b5-96a5-71349c75afda None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:50.587229 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626e5e9a-7e4b-41b5-96a5-71349c75afda None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:50.599477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626e5e9a-7e4b-41b5-96a5-71349c75afda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:50.608649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-626e5e9a-7e4b-41b5-96a5-71349c75afda None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:14:50.608649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:50.608649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:50.608649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:50.608649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:50.608649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:50.608649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:50.608649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:50.608649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:50.608649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:50.608649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:50.608649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:50.608649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:50.608649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:50.608649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:50.608649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:50.609630 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:50.609630 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:50.609630 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:50.609630 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:50.609630 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:50.609630 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:50.609630 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:50.609630 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:50.609630 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:50.609630 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:50.609630 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:50.609630 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:14:50.609630 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:50.609630 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:50.609630 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:50.609630 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:50.610427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:50.610427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:50.610427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:50.610427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:50.610427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:50.610427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:50.610427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:50.610427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:50.610427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:50.610427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:50.610427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:14:50.610427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:14:50.612952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 177/349] 10.222.0.24 () {56 vars in 1114 bytes} [Fri May 27 03:14:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:50.613997 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e5e9a-7e4b-41b5-96a5-71349c75afda None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:50.620040 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 173/350] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:14:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:14:50.621601 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76fa47-cee5-4312-b14e-01f2de726a40 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:50.643683 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:50.723994 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626e5e9a-7e4b-41b5-96a5-71349c75afda None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:50.732652 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76fa47-cee5-4312-b14e-01f2de726a40 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:50.761841 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6013cdcc-989a-4313-9744-83fc62f2cc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:50.763101 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6013cdcc-989a-4313-9744-83fc62f2cc0b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:50.763585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6013cdcc-989a-4313-9744-83fc62f2cc0b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:50.764068 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6013cdcc-989a-4313-9744-83fc62f2cc0b None admin] PATH_INFO: `/v3/groups/01e533eb757846d9927b53ab53f7edf5` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:50.768973 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4fbbf0-168a-49fb-8a76-8b6c4fe817cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:50.769660 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4fbbf0-168a-49fb-8a76-8b6c4fe817cb None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:50.769945 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4fbbf0-168a-49fb-8a76-8b6c4fe817cb None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:50.770225 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4fbbf0-168a-49fb-8a76-8b6c4fe817cb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:50.772415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6013cdcc-989a-4313-9744-83fc62f2cc0b None admin] RBAC: Authorizing `identity:get_group(group_id=01e533eb757846d9927b53ab53f7edf5)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:50.774090 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6013cdcc-989a-4313-9744-83fc62f2cc0b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:50.776026 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 178/351] 10.222.0.24 () {56 vars in 1180 bytes} [Fri May 27 03:14:50 2022] GET /identity/v3/groups/01e533eb757846d9927b53ab53f7edf5 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:50.777719 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6013cdcc-989a-4313-9744-83fc62f2cc0b None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:50.782265 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4fbbf0-168a-49fb-8a76-8b6c4fe817cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:50.790285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-dc4fbbf0-168a-49fb-8a76-8b6c4fe817cb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:14:50.790285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:50.790285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:50.790285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:50.790285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:50.790285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:50.790285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:50.790285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:50.790285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:50.790285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:50.790285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:50.790285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:50.790285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:50.790285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:50.790285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:50.790285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:50.791240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:50.791240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:50.791240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:50.791240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:50.791240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:50.791240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:50.791240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:50.791240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:50.791240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:50.791240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:50.791240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:50.791240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:14:50.791240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:50.791240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:50.791240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:50.791240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:50.792017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:50.792017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:50.792017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:50.792017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:50.792017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:50.792017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:50.792017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:50.792017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:50.792017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:50.792017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:50.792017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:14:50.792017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:14:50.792017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 174/352] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:50.792686 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4fbbf0-168a-49fb-8a76-8b6c4fe817cb None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:50.811047 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:50.879182 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:50.896923 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6013cdcc-989a-4313-9744-83fc62f2cc0b None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:50.929052 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4fbbf0-168a-49fb-8a76-8b6c4fe817cb None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:50.945024 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0342abe-1eb3-4f3b-be7e-248935648ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:50.946315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0342abe-1eb3-4f3b-be7e-248935648ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:50.946315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0342abe-1eb3-4f3b-be7e-248935648ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:50.946500 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0342abe-1eb3-4f3b-be7e-248935648ec7 None admin] PATH_INFO: `/v3/projects/d95eae40782d41759de176dea214eb16` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:50.949025 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0342abe-1eb3-4f3b-be7e-248935648ec7 None admin] RBAC: Authorizing `identity:get_project(project_id=d95eae40782d41759de176dea214eb16)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:50.950113 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0342abe-1eb3-4f3b-be7e-248935648ec7 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:50.952140 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 179/353] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:50 2022] GET /identity/v3/projects/d95eae40782d41759de176dea214eb16 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:50.952917 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0342abe-1eb3-4f3b-be7e-248935648ec7 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:50.983279 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c289f73-fc06-4738-96fa-504b957eaea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:50.983875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c289f73-fc06-4738-96fa-504b957eaea8 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:50.984260 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c289f73-fc06-4738-96fa-504b957eaea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:50.984438 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c289f73-fc06-4738-96fa-504b957eaea8 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:50.985180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c289f73-fc06-4738-96fa-504b957eaea8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:50.985496 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c289f73-fc06-4738-96fa-504b957eaea8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:50.987760 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c289f73-fc06-4738-96fa-504b957eaea8 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:50.997613 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 175/354] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:14:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:50.999151 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c289f73-fc06-4738-96fa-504b957eaea8 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:51.084332 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0342abe-1eb3-4f3b-be7e-248935648ec7 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:51.109707 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:51.131589 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96f493-e24f-4ac5-90e2-629205379643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:51.133188 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96f493-e24f-4ac5-90e2-629205379643 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:51.133188 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96f493-e24f-4ac5-90e2-629205379643 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:51.133188 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96f493-e24f-4ac5-90e2-629205379643 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:51.133938 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96f493-e24f-4ac5-90e2-629205379643 None admin] RBAC: Adding query filter params (group.id=01e533eb757846d9927b53ab53f7edf5, role.id=343337c6f25943a9ad412047b70ddefa, scope.project.id=9dbc862abb4247f69fc1a65f462d65ce) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:51.134118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96f493-e24f-4ac5-90e2-629205379643 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:51.135664 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96f493-e24f-4ac5-90e2-629205379643 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:51.143532 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 180/355] 10.222.0.24 () {56 vars in 1376 bytes} [Fri May 27 03:14:50 2022] GET /identity/v3/role_assignments?group.id=01e533eb757846d9927b53ab53f7edf5&scope.project.id=9dbc862abb4247f69fc1a65f462d65ce&role.id=343337c6f25943a9ad412047b70ddefa => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:51.145965 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd499cff-3353-4fce-ab82-395ea185535e None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:51.146258 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd499cff-3353-4fce-ab82-395ea185535e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:51.146537 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd499cff-3353-4fce-ab82-395ea185535e None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:51.148368 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 181/356] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:51.148826 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd499cff-3353-4fce-ab82-395ea185535e None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:51.155119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c289f73-fc06-4738-96fa-504b957eaea8 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:51.177155 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:51.207609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de029614-ffff-4f61-bfe8-3a8573b33b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:51.208701 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de029614-ffff-4f61-bfe8-3a8573b33b29 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:51.209776 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de029614-ffff-4f61-bfe8-3a8573b33b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:51.210270 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de029614-ffff-4f61-bfe8-3a8573b33b29 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:51.211793 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de029614-ffff-4f61-bfe8-3a8573b33b29 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:51.212346 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de029614-ffff-4f61-bfe8-3a8573b33b29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:51.213999 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de029614-ffff-4f61-bfe8-3a8573b33b29 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:51.225521 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 176/357] 10.222.0.24 () {56 vars in 1101 bytes} [Fri May 27 03:14:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:51.226746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de029614-ffff-4f61-bfe8-3a8573b33b29 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:51.285666 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd499cff-3353-4fce-ab82-395ea185535e None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:51.335667 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07af84b-175b-4d2c-9b90-b30752e1b488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:51.337693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07af84b-175b-4d2c-9b90-b30752e1b488 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:51.337954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07af84b-175b-4d2c-9b90-b30752e1b488 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:51.338233 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07af84b-175b-4d2c-9b90-b30752e1b488 None admin] PATH_INFO: `/v3/groups/01e533eb757846d9927b53ab53f7edf5` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:51.339987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07af84b-175b-4d2c-9b90-b30752e1b488 None admin] RBAC: Authorizing `identity:get_group(group_id=01e533eb757846d9927b53ab53f7edf5)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:51.342198 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07af84b-175b-4d2c-9b90-b30752e1b488 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:51.343859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 182/358] 10.222.0.24 () {56 vars in 1180 bytes} [Fri May 27 03:14:51 2022] GET /identity/v3/groups/01e533eb757846d9927b53ab53f7edf5 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:51.344582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833dd7cd-12de-41ca-839b-69041aae9e87 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:51.344866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833dd7cd-12de-41ca-839b-69041aae9e87 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:51.345141 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833dd7cd-12de-41ca-839b-69041aae9e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:51.363983 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-833dd7cd-12de-41ca-839b-69041aae9e87 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:51.364718 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-833dd7cd-12de-41ca-839b-69041aae9e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:51.368785 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de029614-ffff-4f61-bfe8-3a8573b33b29 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:51.406880 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412f0c0-fc92-414c-a21f-cb6138fce837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:51.407910 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412f0c0-fc92-414c-a21f-cb6138fce837 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:51.408383 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412f0c0-fc92-414c-a21f-cb6138fce837 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:51.408873 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412f0c0-fc92-414c-a21f-cb6138fce837 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:51.413314 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:51.444139 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 183/359] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:51.446010 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833dd7cd-12de-41ca-839b-69041aae9e87 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:51.469737 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:51.507710 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412f0c0-fc92-414c-a21f-cb6138fce837 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:51.509569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412f0c0-fc92-414c-a21f-cb6138fce837 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:51.579647 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833dd7cd-12de-41ca-839b-69041aae9e87 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:51.627225 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab7926-b8de-48dc-84e4-19b3be89032f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:51.628095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab7926-b8de-48dc-84e4-19b3be89032f None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:51.628376 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab7926-b8de-48dc-84e4-19b3be89032f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:51.628646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab7926-b8de-48dc-84e4-19b3be89032f None admin] PATH_INFO: `/v3/users/af7b00cbaabc44b68c56e8a29c253295` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:51.629881 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab7926-b8de-48dc-84e4-19b3be89032f None admin] RBAC: Authorizing `identity:get_user(user_id=af7b00cbaabc44b68c56e8a29c253295)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:51.631369 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab7926-b8de-48dc-84e4-19b3be89032f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:51.633000 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 184/360] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:14:51 2022] GET /identity/v3/users/af7b00cbaabc44b68c56e8a29c253295 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:51.634892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8f711-4f47-4567-87f0-777640d5c478 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:51.634892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8f711-4f47-4567-87f0-777640d5c478 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:51.634892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8f711-4f47-4567-87f0-777640d5c478 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:51.637238 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 185/361] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:51.637843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8f711-4f47-4567-87f0-777640d5c478 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:51.642714 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 177/362] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:14:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:14:51.643761 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412f0c0-fc92-414c-a21f-cb6138fce837 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:51.695614 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:51.744328 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:51.774906 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8f711-4f47-4567-87f0-777640d5c478 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:51.803855 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412f0c0-fc92-414c-a21f-cb6138fce837 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:51.826357 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb049ecf-06b8-4a91-9bf8-3120c95c893a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:51.827672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb049ecf-06b8-4a91-9bf8-3120c95c893a None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:51.827954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb049ecf-06b8-4a91-9bf8-3120c95c893a None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:51.828227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb049ecf-06b8-4a91-9bf8-3120c95c893a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:51.829069 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb049ecf-06b8-4a91-9bf8-3120c95c893a None admin] RBAC: Adding query filter params (group.id=01e533eb757846d9927b53ab53f7edf5, role.id=343337c6f25943a9ad412047b70ddefa, scope.project.id=d95eae40782d41759de176dea214eb16) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:51.829379 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb049ecf-06b8-4a91-9bf8-3120c95c893a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:51.831390 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb049ecf-06b8-4a91-9bf8-3120c95c893a None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:51.836862 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 186/363] 10.222.0.24 () {56 vars in 1376 bytes} [Fri May 27 03:14:51 2022] GET /identity/v3/role_assignments?group.id=01e533eb757846d9927b53ab53f7edf5&scope.project.id=d95eae40782d41759de176dea214eb16&role.id=343337c6f25943a9ad412047b70ddefa => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:51.837676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb049ecf-06b8-4a91-9bf8-3120c95c893a None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:51.861283 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436a459-313f-4a7e-af29-4931a0bab1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:51.862299 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436a459-313f-4a7e-af29-4931a0bab1b5 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:51.862589 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436a459-313f-4a7e-af29-4931a0bab1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:51.862908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436a459-313f-4a7e-af29-4931a0bab1b5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:51.875873 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a436a459-313f-4a7e-af29-4931a0bab1b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:51.888573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-a436a459-313f-4a7e-af29-4931a0bab1b5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:14:51.888573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:51.888573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:51.888573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:51.888573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:51.888573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:51.888573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:51.888573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:51.888573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:51.888573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:51.888573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:51.888573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:51.888573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:51.888573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:51.888573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:51.888573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:51.889537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:51.889537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:51.889537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:51.889537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:51.889537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:51.889537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:51.889537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:51.889537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:51.889537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:51.889537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:51.889537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:51.889537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:14:51.889537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:51.889537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:51.889537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:51.889537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:51.890337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:51.890337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:51.890337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:51.890337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:51.890337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:51.890337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:51.890337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:51.890337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:51.890337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:51.890337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:51.890337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:14:51.890337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:14:51.893451 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 178/364] 10.222.0.24 () {56 vars in 1114 bytes} [Fri May 27 03:14:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:51.894998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a439177-f0a4-4ea3-9740-5310d9b6462b None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:51.895427 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a439177-f0a4-4ea3-9740-5310d9b6462b None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:51.895867 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a439177-f0a4-4ea3-9740-5310d9b6462b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:51.917503 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-2a439177-f0a4-4ea3-9740-5310d9b6462b None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:51.920110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-2a439177-f0a4-4ea3-9740-5310d9b6462b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:51.973037 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb049ecf-06b8-4a91-9bf8-3120c95c893a None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:52.014912 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 179/365] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:52.016239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a439177-f0a4-4ea3-9740-5310d9b6462b None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:52.023112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ccc764-16b6-4f2a-aa63-a26475a2bf27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:52.023791 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ccc764-16b6-4f2a-aa63-a26475a2bf27 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:52.024072 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ccc764-16b6-4f2a-aa63-a26475a2bf27 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:52.024344 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ccc764-16b6-4f2a-aa63-a26475a2bf27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:52.025190 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ccc764-16b6-4f2a-aa63-a26475a2bf27 None admin] RBAC: Adding query filter params (role.id=aa0d3c6b36a24a718f88d0f4309d26d9, user.id=af7b00cbaabc44b68c56e8a29c253295, scope.system=all) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:52.025503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ccc764-16b6-4f2a-aa63-a26475a2bf27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:52.027710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ccc764-16b6-4f2a-aa63-a26475a2bf27 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:52.045711 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 187/366] 10.222.0.24 () {56 vars in 1308 bytes} [Fri May 27 03:14:51 2022] GET /identity/v3/role_assignments?user.id=af7b00cbaabc44b68c56e8a29c253295&scope.system=all&role.id=aa0d3c6b36a24a718f88d0f4309d26d9 => generated 510 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:52.046648 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ccc764-16b6-4f2a-aa63-a26475a2bf27 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:52.060806 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:52.065665 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:52.190991 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a439177-f0a4-4ea3-9740-5310d9b6462b None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:52.229997 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ccc764-16b6-4f2a-aa63-a26475a2bf27 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:52.231693 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a09f7e-4b12-4e6c-a8c6-f0e887632270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:52.232852 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a09f7e-4b12-4e6c-a8c6-f0e887632270 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:52.233338 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a09f7e-4b12-4e6c-a8c6-f0e887632270 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:52.233809 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a09f7e-4b12-4e6c-a8c6-f0e887632270 None admin] PATH_INFO: `/v3/projects/9dbc862abb4247f69fc1a65f462d65ce` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:52.235295 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a09f7e-4b12-4e6c-a8c6-f0e887632270 None admin] RBAC: Authorizing `identity:get_project(project_id=9dbc862abb4247f69fc1a65f462d65ce)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:52.237045 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a09f7e-4b12-4e6c-a8c6-f0e887632270 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:52.239621 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 180/367] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:52 2022] GET /identity/v3/projects/9dbc862abb4247f69fc1a65f462d65ce => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:52.240599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a09f7e-4b12-4e6c-a8c6-f0e887632270 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:52.276114 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481143a0-0823-48c9-b46e-4fbc47c52118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:52.277094 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481143a0-0823-48c9-b46e-4fbc47c52118 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:52.277394 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481143a0-0823-48c9-b46e-4fbc47c52118 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:52.277671 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481143a0-0823-48c9-b46e-4fbc47c52118 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:52.278950 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481143a0-0823-48c9-b46e-4fbc47c52118 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:52.279271 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481143a0-0823-48c9-b46e-4fbc47c52118 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:52.280949 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481143a0-0823-48c9-b46e-4fbc47c52118 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:52.292509 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 188/368] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:14:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:52.293425 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481143a0-0823-48c9-b46e-4fbc47c52118 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:52.360433 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:52.371381 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a09f7e-4b12-4e6c-a8c6-f0e887632270 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:52.384917 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:52.428278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dc6ab4-3096-4403-82c3-478a31758d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:52.429364 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc6ab4-3096-4403-82c3-478a31758d7d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:52.429648 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc6ab4-3096-4403-82c3-478a31758d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:52.429920 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dc6ab4-3096-4403-82c3-478a31758d7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:52.449465 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dc6ab4-3096-4403-82c3-478a31758d7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:52.456210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-a2dc6ab4-3096-4403-82c3-478a31758d7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:14:52.456210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:52.456210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:52.456210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:52.456210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:52.456210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:52.456210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:52.456210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:52.456210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:52.456210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:52.456210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:52.456210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:52.456210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:52.456210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:52.456210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:52.456210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:52.457099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:52.457099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:52.457099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:52.457099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:52.457099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:52.457099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:52.457099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:52.457099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:52.457099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:52.457099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:52.457099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:52.457099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:14:52.457099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:52.457099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:52.457099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:52.457099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:52.457876 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:52.457876 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:52.457876 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:52.457876 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:52.457876 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:52.457876 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:52.457876 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:52.457876 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:52.457876 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:52.457876 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:52.457876 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:14:52.457876 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:14:52.457876 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 181/369] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:14:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:52.458536 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dc6ab4-3096-4403-82c3-478a31758d7d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:52.459445 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481143a0-0823-48c9-b46e-4fbc47c52118 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:52.509952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b64246-6b08-46b9-871e-60a21672711e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:52.511098 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b64246-6b08-46b9-871e-60a21672711e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:52.512182 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b64246-6b08-46b9-871e-60a21672711e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:52.512617 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b64246-6b08-46b9-871e-60a21672711e None admin] PATH_INFO: `/v3/groups/01e533eb757846d9927b53ab53f7edf5` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:52.514040 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b64246-6b08-46b9-871e-60a21672711e None admin] RBAC: Authorizing `identity:get_group(group_id=01e533eb757846d9927b53ab53f7edf5)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:52.516243 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b64246-6b08-46b9-871e-60a21672711e None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:52.518878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 189/370] 10.222.0.24 () {56 vars in 1180 bytes} [Fri May 27 03:14:52 2022] GET /identity/v3/groups/01e533eb757846d9927b53ab53f7edf5 => generated 226 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:52.520629 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b64246-6b08-46b9-871e-60a21672711e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:52.588914 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dc6ab4-3096-4403-82c3-478a31758d7d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:52.647472 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b64246-6b08-46b9-871e-60a21672711e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:52.648643 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f5c51-ab78-4c3f-a37d-0ceb544342ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:52.651460 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f5c51-ab78-4c3f-a37d-0ceb544342ab None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:52.651460 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f5c51-ab78-4c3f-a37d-0ceb544342ab None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:52.651460 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f5c51-ab78-4c3f-a37d-0ceb544342ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:52.680335 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:52.696271 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7624854e-0cb9-4b31-902c-2e45b1c42cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:52.697172 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624854e-0cb9-4b31-902c-2e45b1c42cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:52.701935 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624854e-0cb9-4b31-902c-2e45b1c42cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:52.701935 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7624854e-0cb9-4b31-902c-2e45b1c42cd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:52.701935 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7624854e-0cb9-4b31-902c-2e45b1c42cd3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:52.701935 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7624854e-0cb9-4b31-902c-2e45b1c42cd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:52.701935 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7624854e-0cb9-4b31-902c-2e45b1c42cd3 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:52.704140 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:52.711661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 190/371] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:52.712843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7624854e-0cb9-4b31-902c-2e45b1c42cd3 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:52.740824 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f5c51-ab78-4c3f-a37d-0ceb544342ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:52.742874 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f5c51-ab78-4c3f-a37d-0ceb544342ab None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:52.847622 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7624854e-0cb9-4b31-902c-2e45b1c42cd3 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:52.879711 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 182/372] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:14:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:14:52.881327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f5c51-ab78-4c3f-a37d-0ceb544342ab None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:52.885499 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994faea-6e02-4ab5-bfb9-6806676dddf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:52.887562 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994faea-6e02-4ab5-bfb9-6806676dddf8 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:52.888042 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994faea-6e02-4ab5-bfb9-6806676dddf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:52.888519 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994faea-6e02-4ab5-bfb9-6806676dddf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:52.889596 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994faea-6e02-4ab5-bfb9-6806676dddf8 None admin] RBAC: Adding query filter params (group.id=01e533eb757846d9927b53ab53f7edf5, role.id=d7133ce917f0431883775551ab086d5d, scope.project.id=9dbc862abb4247f69fc1a65f462d65ce) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:52.890094 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994faea-6e02-4ab5-bfb9-6806676dddf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:52.891735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994faea-6e02-4ab5-bfb9-6806676dddf8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:52.897827 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 191/373] 10.222.0.24 () {56 vars in 1376 bytes} [Fri May 27 03:14:52 2022] GET /identity/v3/role_assignments?group.id=01e533eb757846d9927b53ab53f7edf5&scope.project.id=9dbc862abb4247f69fc1a65f462d65ce&role.id=d7133ce917f0431883775551ab086d5d => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:52.898934 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994faea-6e02-4ab5-bfb9-6806676dddf8 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:52.954292 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:52.982326 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:53.049662 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f5c51-ab78-4c3f-a37d-0ceb544342ab None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:53.057312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994faea-6e02-4ab5-bfb9-6806676dddf8 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:53.112267 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40652e3-0d53-4b86-94ba-319d9993d897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:53.113357 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40652e3-0d53-4b86-94ba-319d9993d897 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:53.113844 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40652e3-0d53-4b86-94ba-319d9993d897 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:53.114321 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40652e3-0d53-4b86-94ba-319d9993d897 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:53.154290 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71626042-1f58-4540-a697-5b0bf1cbc621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:53.155552 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71626042-1f58-4540-a697-5b0bf1cbc621 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:53.155840 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71626042-1f58-4540-a697-5b0bf1cbc621 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:53.156119 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71626042-1f58-4540-a697-5b0bf1cbc621 None admin] PATH_INFO: `/v3/projects/d95eae40782d41759de176dea214eb16` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:53.160462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71626042-1f58-4540-a697-5b0bf1cbc621 None admin] RBAC: Authorizing `identity:get_project(project_id=d95eae40782d41759de176dea214eb16)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:53.161166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71626042-1f58-4540-a697-5b0bf1cbc621 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:53.163001 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 192/374] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:52 2022] GET /identity/v3/projects/d95eae40782d41759de176dea214eb16 => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:53.174163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71626042-1f58-4540-a697-5b0bf1cbc621 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:53.206562 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40652e3-0d53-4b86-94ba-319d9993d897 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:53.208443 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40652e3-0d53-4b86-94ba-319d9993d897 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:53.267893 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:53.345522 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 183/375] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:14:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:14:53.351135 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40652e3-0d53-4b86-94ba-319d9993d897 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:53.398715 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71626042-1f58-4540-a697-5b0bf1cbc621 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:53.447193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:53.453182 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4842d9-2863-4eda-88ab-b887428c58c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:53.454375 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4842d9-2863-4eda-88ab-b887428c58c7 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:53.454963 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4842d9-2863-4eda-88ab-b887428c58c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:53.464465 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4842d9-2863-4eda-88ab-b887428c58c7 None admin] PATH_INFO: `/v3/groups/01e533eb757846d9927b53ab53f7edf5` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:53.464465 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4842d9-2863-4eda-88ab-b887428c58c7 None admin] RBAC: Authorizing `identity:get_group(group_id=01e533eb757846d9927b53ab53f7edf5)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:53.464465 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4842d9-2863-4eda-88ab-b887428c58c7 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:53.464465 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 193/376] 10.222.0.24 () {56 vars in 1180 bytes} [Fri May 27 03:14:53 2022] GET /identity/v3/groups/01e533eb757846d9927b53ab53f7edf5 => generated 226 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:53.467982 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4842d9-2863-4eda-88ab-b887428c58c7 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:53.558498 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40652e3-0d53-4b86-94ba-319d9993d897 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:53.584078 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4842d9-2863-4eda-88ab-b887428c58c7 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:53.596706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37603bc-3d93-4b49-8eab-e11ffc3f8398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:53.597845 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37603bc-3d93-4b49-8eab-e11ffc3f8398 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:53.598008 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37603bc-3d93-4b49-8eab-e11ffc3f8398 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:53.598384 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37603bc-3d93-4b49-8eab-e11ffc3f8398 None admin] PATH_INFO: `/v3/projects/4d3e8df9c2aa4b4588906f73830bd805` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:53.599855 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37603bc-3d93-4b49-8eab-e11ffc3f8398 None admin] RBAC: Authorizing `identity:get_project(project_id=4d3e8df9c2aa4b4588906f73830bd805)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:53.601500 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37603bc-3d93-4b49-8eab-e11ffc3f8398 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:53.603794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 184/377] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:53 2022] GET /identity/v3/projects/4d3e8df9c2aa4b4588906f73830bd805 => generated 344 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:53.609960 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37603bc-3d93-4b49-8eab-e11ffc3f8398 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:53.627510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7701c2-38af-4b8b-9416-b977d9cec6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:53.628479 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7701c2-38af-4b8b-9416-b977d9cec6fa None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:53.628922 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7701c2-38af-4b8b-9416-b977d9cec6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:53.629373 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7701c2-38af-4b8b-9416-b977d9cec6fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:53.630383 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7701c2-38af-4b8b-9416-b977d9cec6fa None admin] RBAC: Adding query filter params (group.id=01e533eb757846d9927b53ab53f7edf5, role.id=d7133ce917f0431883775551ab086d5d, scope.project.id=d95eae40782d41759de176dea214eb16) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:53.630922 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7701c2-38af-4b8b-9416-b977d9cec6fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:53.632598 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7701c2-38af-4b8b-9416-b977d9cec6fa None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:53.646975 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 194/378] 10.222.0.24 () {56 vars in 1376 bytes} [Fri May 27 03:14:53 2022] GET /identity/v3/role_assignments?group.id=01e533eb757846d9927b53ab53f7edf5&scope.project.id=d95eae40782d41759de176dea214eb16&role.id=d7133ce917f0431883775551ab086d5d => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:53.671253 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:53.731105 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37603bc-3d93-4b49-8eab-e11ffc3f8398 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:53.769722 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e83b44f-0a22-4a74-9297-16d1498eef70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:53.770612 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83b44f-0a22-4a74-9297-16d1498eef70 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:53.770923 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83b44f-0a22-4a74-9297-16d1498eef70 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:53.771204 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83b44f-0a22-4a74-9297-16d1498eef70 None admin] PATH_INFO: `/v3/groups/6246a690b6ed4faf896eba90272c3265` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:53.778277 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e83b44f-0a22-4a74-9297-16d1498eef70 None admin] RBAC: Authorizing `identity:get_group(group_id=6246a690b6ed4faf896eba90272c3265)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:53.779854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e83b44f-0a22-4a74-9297-16d1498eef70 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:53.781924 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 185/379] 10.222.0.24 () {56 vars in 1180 bytes} [Fri May 27 03:14:53 2022] GET /identity/v3/groups/6246a690b6ed4faf896eba90272c3265 => generated 229 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:53.787840 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7701c2-38af-4b8b-9416-b977d9cec6fa None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:53.827454 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee3c8a-4581-4cd0-bfe2-767f92965f77 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:53.827794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee3c8a-4581-4cd0-bfe2-767f92965f77 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:53.828016 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee3c8a-4581-4cd0-bfe2-767f92965f77 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:53.830057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 186/380] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:53.849991 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607574e3-a25b-4f8a-ac85-ab0d53e14ff9 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:53.850321 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607574e3-a25b-4f8a-ac85-ab0d53e14ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:53.850598 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607574e3-a25b-4f8a-ac85-ab0d53e14ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:53.875505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-607574e3-a25b-4f8a-ac85-ab0d53e14ff9 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:53.877177 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-607574e3-a25b-4f8a-ac85-ab0d53e14ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:53.900146 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:53.944751 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7701c2-38af-4b8b-9416-b977d9cec6fa None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:53.960254 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 187/381] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:53.983259 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607574e3-a25b-4f8a-ac85-ab0d53e14ff9 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:53.985373 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f0004-2b49-422a-bb16-0d45ec5fb279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:53.986521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f0004-2b49-422a-bb16-0d45ec5fb279 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:53.987062 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f0004-2b49-422a-bb16-0d45ec5fb279 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:53.987550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f0004-2b49-422a-bb16-0d45ec5fb279 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:53.988596 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f0004-2b49-422a-bb16-0d45ec5fb279 None admin] RBAC: Adding query filter params (group.id=6246a690b6ed4faf896eba90272c3265, role.id=e5a908bd3ed5445b82a6053246533b88, scope.project.id=4d3e8df9c2aa4b4588906f73830bd805) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:53.989115 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f0004-2b49-422a-bb16-0d45ec5fb279 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:53.990997 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890f0004-2b49-422a-bb16-0d45ec5fb279 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:54.000073 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 195/382] 10.222.0.24 () {56 vars in 1376 bytes} [Fri May 27 03:14:53 2022] GET /identity/v3/role_assignments?group.id=6246a690b6ed4faf896eba90272c3265&scope.project.id=4d3e8df9c2aa4b4588906f73830bd805&role.id=e5a908bd3ed5445b82a6053246533b88 => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:54.009643 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:54.109843 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607574e3-a25b-4f8a-ac85-ab0d53e14ff9 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:54.150063 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea260f6d-cb3c-43f7-85a6-68b3b05beaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:54.150902 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea260f6d-cb3c-43f7-85a6-68b3b05beaf2 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:54.151185 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea260f6d-cb3c-43f7-85a6-68b3b05beaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:54.151458 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea260f6d-cb3c-43f7-85a6-68b3b05beaf2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:54.163291 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea260f6d-cb3c-43f7-85a6-68b3b05beaf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:54.171908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-ea260f6d-cb3c-43f7-85a6-68b3b05beaf2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:54.171908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:54.171908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:54.171908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:54.171908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:54.171908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:54.171908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:54.171908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:54.171908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:54.171908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:54.171908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:54.171908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:54.171908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:54.171908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:54.171908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:54.171908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:54.172842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:54.172842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:54.172842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:54.172842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:54.172842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:54.172842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:54.172842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:54.172842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:54.172842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:54.172842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:54.172842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:54.172842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:14:54.172842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:54.172842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:54.172842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:54.172842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:54.173791 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:54.173791 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:54.173791 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:54.173791 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:54.173791 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:54.173791 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:54.173791 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:54.173791 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:54.173791 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:54.173791 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:54.173791 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:54.173791 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:14:54.173791 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 188/383] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:14:54.178900 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f0004-2b49-422a-bb16-0d45ec5fb279 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:54.285242 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890f0004-2b49-422a-bb16-0d45ec5fb279 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:54.317892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:54.320168 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391377c-3f7f-45e2-be00-b892b205c542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:54.320816 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391377c-3f7f-45e2-be00-b892b205c542 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:54.321133 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391377c-3f7f-45e2-be00-b892b205c542 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:54.321418 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391377c-3f7f-45e2-be00-b892b205c542 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:54.322172 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f391377c-3f7f-45e2-be00-b892b205c542 None admin] RBAC: Adding query filter params (name=member) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:54.322730 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f391377c-3f7f-45e2-be00-b892b205c542 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:54.324753 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f391377c-3f7f-45e2-be00-b892b205c542 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:54.334530 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 196/384] 10.222.0.24 () {56 vars in 1101 bytes} [Fri May 27 03:14:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:54.340683 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea260f6d-cb3c-43f7-85a6-68b3b05beaf2 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:54.423200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:54.450548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea260f6d-cb3c-43f7-85a6-68b3b05beaf2 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:54.486257 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4c6c7-d68f-4761-8f87-d06dbade501b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:54.487443 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4c6c7-d68f-4761-8f87-d06dbade501b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:54.487919 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4c6c7-d68f-4761-8f87-d06dbade501b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:54.488360 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4c6c7-d68f-4761-8f87-d06dbade501b None admin] PATH_INFO: `/v3/groups/01e533eb757846d9927b53ab53f7edf5` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:54.489740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4c6c7-d68f-4761-8f87-d06dbade501b None admin] RBAC: Authorizing `identity:get_group(group_id=01e533eb757846d9927b53ab53f7edf5)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:54.491414 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4c6c7-d68f-4761-8f87-d06dbade501b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:54.493521 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 189/385] 10.222.0.24 () {56 vars in 1180 bytes} [Fri May 27 03:14:54 2022] GET /identity/v3/groups/01e533eb757846d9927b53ab53f7edf5 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:54.499102 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391377c-3f7f-45e2-be00-b892b205c542 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:54.604129 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391377c-3f7f-45e2-be00-b892b205c542 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:54.640154 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5242a-bb6d-4893-90cb-89342f0da058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:54.641351 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5242a-bb6d-4893-90cb-89342f0da058 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:54.641826 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5242a-bb6d-4893-90cb-89342f0da058 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:54.642281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5242a-bb6d-4893-90cb-89342f0da058 None admin] PATH_INFO: `/v3/projects/9dbc862abb4247f69fc1a65f462d65ce` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:54.643656 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5242a-bb6d-4893-90cb-89342f0da058 None admin] RBAC: Authorizing `identity:get_project(project_id=9dbc862abb4247f69fc1a65f462d65ce)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:54.645257 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5242a-bb6d-4893-90cb-89342f0da058 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:54.647451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 197/386] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:54 2022] GET /identity/v3/projects/9dbc862abb4247f69fc1a65f462d65ce => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:54.655599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4c6c7-d68f-4761-8f87-d06dbade501b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:54.778989 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4c6c7-d68f-4761-8f87-d06dbade501b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:54.821459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082d39c-8a58-4a4f-957f-5cd3888600bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:54.822356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082d39c-8a58-4a4f-957f-5cd3888600bd None admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:54.822634 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082d39c-8a58-4a4f-957f-5cd3888600bd None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:54.822935 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082d39c-8a58-4a4f-957f-5cd3888600bd None admin] PATH_INFO: `/v3/projects/9dbc862abb4247f69fc1a65f462d65ce/groups/01e533eb757846d9927b53ab53f7edf5/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:54.825239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2082d39c-8a58-4a4f-957f-5cd3888600bd None admin] RBAC: Authorizing `identity:create_grant(project_id=9dbc862abb4247f69fc1a65f462d65ce, group_id=01e533eb757846d9927b53ab53f7edf5, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:54.827045 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2082d39c-8a58-4a4f-957f-5cd3888600bd None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:54.922729 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 190/387] 10.222.0.24 () {58 vars in 1442 bytes} [Fri May 27 03:14:54 2022] PUT /identity/v3/projects/9dbc862abb4247f69fc1a65f462d65ce/groups/01e533eb757846d9927b53ab53f7edf5/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:14:55.036890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291887e-c979-458d-8e52-f3492d83a4b8 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:55.036890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291887e-c979-458d-8e52-f3492d83a4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:55.036890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291887e-c979-458d-8e52-f3492d83a4b8 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:55.038262 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 198/388] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:55.044885 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf33935-f684-4384-afeb-d2b0e1f66b1c None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:55.045181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf33935-f684-4384-afeb-d2b0e1f66b1c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:55.045440 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf33935-f684-4384-afeb-d2b0e1f66b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:55.054041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:55.072039 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-fdf33935-f684-4384-afeb-d2b0e1f66b1c None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:55.073318 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-fdf33935-f684-4384-afeb-d2b0e1f66b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:55.175702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 191/389] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:55.207122 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291887e-c979-458d-8e52-f3492d83a4b8 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:55.291307 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:55.347520 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291887e-c979-458d-8e52-f3492d83a4b8 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:55.389878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c8189-6b8b-4d73-bc34-bc005280cf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:55.390685 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c8189-6b8b-4d73-bc34-bc005280cf82 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:55.390986 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c8189-6b8b-4d73-bc34-bc005280cf82 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:55.391256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c8189-6b8b-4d73-bc34-bc005280cf82 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:55.412154 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c8189-6b8b-4d73-bc34-bc005280cf82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:55.420003 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-bc4c8189-6b8b-4d73-bc34-bc005280cf82 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:55.420003 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:55.420003 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:55.420003 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:55.420003 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:55.420003 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:55.420003 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:55.420003 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:55.420003 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:55.420003 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:55.420003 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:55.420003 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:55.420003 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:55.420003 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:55.420003 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:55.420003 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:55.420884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:55.420884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:55.420884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:55.420884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:55.420884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:55.420884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:55.420884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:55.420884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:55.420884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:55.420884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:55.420884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:55.420884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:14:55.420884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:55.420884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:55.420884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:55.420884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:55.422086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:55.422086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:55.422086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:55.422086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:55.422086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:55.422086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:55.422086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:55.422086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:55.422086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:55.422086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:55.422086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:55.422086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:14:55.422086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 199/390] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:55.427999 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf33935-f684-4384-afeb-d2b0e1f66b1c None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:55.576605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf33935-f684-4384-afeb-d2b0e1f66b1c None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:55.614008 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:55.618982 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce874816-2d3b-4517-94fb-c7cd8f5a7cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:55.619657 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce874816-2d3b-4517-94fb-c7cd8f5a7cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:55.619935 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce874816-2d3b-4517-94fb-c7cd8f5a7cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:55.620206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce874816-2d3b-4517-94fb-c7cd8f5a7cd8 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:55.620987 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce874816-2d3b-4517-94fb-c7cd8f5a7cd8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:55.621291 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce874816-2d3b-4517-94fb-c7cd8f5a7cd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:55.622659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce874816-2d3b-4517-94fb-c7cd8f5a7cd8 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:55.634191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 192/391] 10.222.0.24 () {56 vars in 1101 bytes} [Fri May 27 03:14:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:55.640077 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c8189-6b8b-4d73-bc34-bc005280cf82 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:55.833647 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c8189-6b8b-4d73-bc34-bc005280cf82 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:55.874618 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:55.906822 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d10bba-6110-45f0-9351-41de79af5bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:55.908093 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d10bba-6110-45f0-9351-41de79af5bcd None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:55.908666 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d10bba-6110-45f0-9351-41de79af5bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:55.909136 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d10bba-6110-45f0-9351-41de79af5bcd None admin] PATH_INFO: `/v3/groups/01e533eb757846d9927b53ab53f7edf5` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:55.910671 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d10bba-6110-45f0-9351-41de79af5bcd None admin] RBAC: Authorizing `identity:get_group(group_id=01e533eb757846d9927b53ab53f7edf5)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:55.912349 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d10bba-6110-45f0-9351-41de79af5bcd None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:55.914189 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 200/392] 10.222.0.24 () {56 vars in 1180 bytes} [Fri May 27 03:14:55 2022] GET /identity/v3/groups/01e533eb757846d9927b53ab53f7edf5 => generated 226 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:55.919690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce874816-2d3b-4517-94fb-c7cd8f5a7cd8 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:55.952796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f061b-d7aa-41c2-aa76-938aa414c02d None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:55.953118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f061b-d7aa-41c2-aa76-938aa414c02d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:55.953355 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f061b-d7aa-41c2-aa76-938aa414c02d None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:55.955012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 201/393] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:14:55.960221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc8f48-b5e9-400e-b249-4a43745bf1d9 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:55.960509 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc8f48-b5e9-400e-b249-4a43745bf1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:55.960787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dc8f48-b5e9-400e-b249-4a43745bf1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:55.990393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-a8dc8f48-b5e9-400e-b249-4a43745bf1d9 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:55.996842 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a8dc8f48-b5e9-400e-b249-4a43745bf1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:56.096117 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce874816-2d3b-4517-94fb-c7cd8f5a7cd8 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:56.127286 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 202/394] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:14:56.169069 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206d7d0-9813-452f-ae83-72f0842fe888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:56.170336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206d7d0-9813-452f-ae83-72f0842fe888 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:56.170877 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206d7d0-9813-452f-ae83-72f0842fe888 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:56.171369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8206d7d0-9813-452f-ae83-72f0842fe888 None admin] PATH_INFO: `/v3/projects/d95eae40782d41759de176dea214eb16` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:56.172838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8206d7d0-9813-452f-ae83-72f0842fe888 None admin] RBAC: Authorizing `identity:get_project(project_id=d95eae40782d41759de176dea214eb16)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:56.174489 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8206d7d0-9813-452f-ae83-72f0842fe888 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:56.176733 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 193/395] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:55 2022] GET /identity/v3/projects/d95eae40782d41759de176dea214eb16 => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:56.182002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc8f48-b5e9-400e-b249-4a43745bf1d9 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:56.186106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206d7d0-9813-452f-ae83-72f0842fe888 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:56.305296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:56.359348 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8206d7d0-9813-452f-ae83-72f0842fe888 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:56.389001 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dc8f48-b5e9-400e-b249-4a43745bf1d9 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:56.399272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eed9e2-123f-430d-9a92-b0fca39c38fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:56.400473 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eed9e2-123f-430d-9a92-b0fca39c38fd None admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:56.400958 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eed9e2-123f-430d-9a92-b0fca39c38fd None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:56.401885 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eed9e2-123f-430d-9a92-b0fca39c38fd None admin] PATH_INFO: `/v3/projects/d95eae40782d41759de176dea214eb16/groups/01e533eb757846d9927b53ab53f7edf5/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:56.404142 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eed9e2-123f-430d-9a92-b0fca39c38fd None admin] RBAC: Authorizing `identity:create_grant(project_id=d95eae40782d41759de176dea214eb16, group_id=01e533eb757846d9927b53ab53f7edf5, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:56.405808 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eed9e2-123f-430d-9a92-b0fca39c38fd None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:56.419366 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:56.441407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101f949-0f1c-47ac-bebd-8c97936b587c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:56.442612 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101f949-0f1c-47ac-bebd-8c97936b587c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:56.443100 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101f949-0f1c-47ac-bebd-8c97936b587c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:56.443550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101f949-0f1c-47ac-bebd-8c97936b587c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:56.455564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3101f949-0f1c-47ac-bebd-8c97936b587c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:56.464143 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-3101f949-0f1c-47ac-bebd-8c97936b587c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:14:56.464143 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:56.464143 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:56.464143 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:56.464143 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:56.464143 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:56.464143 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:56.464143 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:56.464143 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:56.464143 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:56.464143 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:56.464143 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:56.464143 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:56.464143 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:56.464143 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:56.464143 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:56.467350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:56.467350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:56.467350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:56.467350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:56.467350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:56.467350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:56.467350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:56.467350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:56.467350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:56.467350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:56.467350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:56.467350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:14:56.467350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:56.467350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:56.467350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:56.467350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:56.468147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:56.468147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:56.468147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:56.468147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:56.468147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:56.468147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:56.468147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:56.468147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:56.468147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:56.468147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:56.468147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:14:56.468147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:14:56.468147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 203/396] 10.222.0.24 () {56 vars in 1114 bytes} [Fri May 27 03:14:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:56.472649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101f949-0f1c-47ac-bebd-8c97936b587c None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:56.493726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 194/397] 10.222.0.24 () {58 vars in 1442 bytes} [Fri May 27 03:14:56 2022] PUT /identity/v3/projects/d95eae40782d41759de176dea214eb16/groups/01e533eb757846d9927b53ab53f7edf5/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:14:56.597193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101f949-0f1c-47ac-bebd-8c97936b587c None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:56.635254 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42ac0e-1cf8-4bf0-90df-f6556d8f21dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:56.635864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42ac0e-1cf8-4bf0-90df-f6556d8f21dd None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:56.636672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42ac0e-1cf8-4bf0-90df-f6556d8f21dd None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:56.636965 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42ac0e-1cf8-4bf0-90df-f6556d8f21dd None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:56.637733 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42ac0e-1cf8-4bf0-90df-f6556d8f21dd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:56.638043 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42ac0e-1cf8-4bf0-90df-f6556d8f21dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:56.639489 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42ac0e-1cf8-4bf0-90df-f6556d8f21dd None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:56.649165 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 204/398] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:14:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:56.655477 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eed9e2-123f-430d-9a92-b0fca39c38fd None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:56.768144 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eed9e2-123f-430d-9a92-b0fca39c38fd None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:56.789130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf501b-ba7e-4b9f-944c-6861022feada None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:56.789427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf501b-ba7e-4b9f-944c-6861022feada None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:56.789711 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf501b-ba7e-4b9f-944c-6861022feada None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:56.791155 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 205/399] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:14:56.792773 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:56.797944 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb22d5-f135-44a8-8420-edfb6a651987 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:56.798233 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb22d5-f135-44a8-8420-edfb6a651987 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:56.798517 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fb22d5-f135-44a8-8420-edfb6a651987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:56.805228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932d812-9507-4ba5-9bbd-3d524e9b1c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:56.811644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932d812-9507-4ba5-9bbd-3d524e9b1c9d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:56.811644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932d812-9507-4ba5-9bbd-3d524e9b1c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:56.811644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932d812-9507-4ba5-9bbd-3d524e9b1c9d None admin] PATH_INFO: `/v3/groups/01e533eb757846d9927b53ab53f7edf5` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:56.811644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1932d812-9507-4ba5-9bbd-3d524e9b1c9d None admin] RBAC: Authorizing `identity:get_group(group_id=01e533eb757846d9927b53ab53f7edf5)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:56.811644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1932d812-9507-4ba5-9bbd-3d524e9b1c9d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:56.812175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 195/400] 10.222.0.24 () {56 vars in 1180 bytes} [Fri May 27 03:14:56 2022] GET /identity/v3/groups/01e533eb757846d9927b53ab53f7edf5 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:56.817908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932d812-9507-4ba5-9bbd-3d524e9b1c9d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:56.824095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-72fb22d5-f135-44a8-8420-edfb6a651987 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:56.825326 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-72fb22d5-f135-44a8-8420-edfb6a651987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:56.863551 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:56.904759 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 206/401] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:14:56.926951 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb22d5-f135-44a8-8420-edfb6a651987 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:57.016795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932d812-9507-4ba5-9bbd-3d524e9b1c9d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:57.034748 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fb22d5-f135-44a8-8420-edfb6a651987 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:57.060743 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57bdf8-3c6f-4128-8f65-dc40124a2f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:57.061939 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57bdf8-3c6f-4128-8f65-dc40124a2f05 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:57.062396 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57bdf8-3c6f-4128-8f65-dc40124a2f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:57.062959 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57bdf8-3c6f-4128-8f65-dc40124a2f05 None admin] PATH_INFO: `/v3/projects/9dbc862abb4247f69fc1a65f462d65ce` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:57.064413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef57bdf8-3c6f-4128-8f65-dc40124a2f05 None admin] RBAC: Authorizing `identity:get_project(project_id=9dbc862abb4247f69fc1a65f462d65ce)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:57.066152 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef57bdf8-3c6f-4128-8f65-dc40124a2f05 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:57.068548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 196/402] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:56 2022] GET /identity/v3/projects/9dbc862abb4247f69fc1a65f462d65ce => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:57.070023 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c14708-2555-4586-b42e-fac5bd78ba9d None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:57.070485 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c14708-2555-4586-b42e-fac5bd78ba9d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:57.070982 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c14708-2555-4586-b42e-fac5bd78ba9d None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:57.076467 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7cebc-54f1-4413-bdd9-53d048fd170c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:57.077288 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7cebc-54f1-4413-bdd9-53d048fd170c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:57.077578 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7cebc-54f1-4413-bdd9-53d048fd170c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:57.077852 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7cebc-54f1-4413-bdd9-53d048fd170c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:57.084353 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 197/403] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:57 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:57.085506 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c14708-2555-4586-b42e-fac5bd78ba9d None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:57.090878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7cebc-54f1-4413-bdd9-53d048fd170c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:57.103676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-d6f7cebc-54f1-4413-bdd9-53d048fd170c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:14:57.103676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:57.103676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:57.103676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:57.103676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:57.103676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:57.103676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:57.103676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:57.103676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:57.103676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:57.103676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:57.103676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:57.103676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:57.103676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:57.103676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:57.103676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:57.104556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:57.104556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:57.104556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:57.104556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:57.104556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:57.104556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:57.104556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:57.104556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:57.104556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:57.104556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:57.104556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:57.104556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:14:57.104556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:57.104556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:57.104556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:57.104556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:57.105951 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:57.105951 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:57.105951 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:57.105951 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:57.105951 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:57.105951 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:57.105951 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:57.105951 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:57.105951 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:57.105951 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:57.105951 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:14:57.105951 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:14:57.108635 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 207/404] 10.222.0.24 () {56 vars in 1114 bytes} [Fri May 27 03:14:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:57.110076 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364a957-6585-4b9e-bb58-4ec2dca03978 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:57.110488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364a957-6585-4b9e-bb58-4ec2dca03978 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:57.110916 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364a957-6585-4b9e-bb58-4ec2dca03978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:57.132448 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-d364a957-6585-4b9e-bb58-4ec2dca03978 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:57.135229 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d364a957-6585-4b9e-bb58-4ec2dca03978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:57.162053 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:57.165740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:57.215591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c14708-2555-4586-b42e-fac5bd78ba9d None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:57.224973 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 208/405] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:57.226501 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364a957-6585-4b9e-bb58-4ec2dca03978 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:57.265470 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577829df-4503-4623-8f2b-f1a2827c248f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:57.266357 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577829df-4503-4623-8f2b-f1a2827c248f None admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:57.266638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577829df-4503-4623-8f2b-f1a2827c248f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:57.266956 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577829df-4503-4623-8f2b-f1a2827c248f None admin] PATH_INFO: `/v3/projects/9dbc862abb4247f69fc1a65f462d65ce/groups/01e533eb757846d9927b53ab53f7edf5/roles/d7133ce917f0431883775551ab086d5d` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:57.269218 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577829df-4503-4623-8f2b-f1a2827c248f None admin] RBAC: Authorizing `identity:create_grant(project_id=9dbc862abb4247f69fc1a65f462d65ce, group_id=01e533eb757846d9927b53ab53f7edf5, role_id=d7133ce917f0431883775551ab086d5d)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:57.270691 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577829df-4503-4623-8f2b-f1a2827c248f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:57.359194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364a957-6585-4b9e-bb58-4ec2dca03978 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:57.366987 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 198/406] 10.222.0.24 () {58 vars in 1442 bytes} [Fri May 27 03:14:57 2022] PUT /identity/v3/projects/9dbc862abb4247f69fc1a65f462d65ce/groups/01e533eb757846d9927b53ab53f7edf5/roles/d7133ce917f0431883775551ab086d5d => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:14:57.368048 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577829df-4503-4623-8f2b-f1a2827c248f None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:57.410387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc04c79-8b7a-4a67-8315-f830eb33a5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:57.411567 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc04c79-8b7a-4a67-8315-f830eb33a5e0 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:57.412133 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc04c79-8b7a-4a67-8315-f830eb33a5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:57.412935 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc04c79-8b7a-4a67-8315-f830eb33a5e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:57.413979 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc04c79-8b7a-4a67-8315-f830eb33a5e0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:57.414501 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc04c79-8b7a-4a67-8315-f830eb33a5e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:57.416808 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc04c79-8b7a-4a67-8315-f830eb33a5e0 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:57.428448 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 209/407] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:14:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:57.435967 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc04c79-8b7a-4a67-8315-f830eb33a5e0 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:57.492642 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:57.548033 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577829df-4503-4623-8f2b-f1a2827c248f None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:57.574871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:57.599162 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a9c0e-6836-43b7-ae38-8a340930f906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:57.599989 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a9c0e-6836-43b7-ae38-8a340930f906 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:57.600271 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a9c0e-6836-43b7-ae38-8a340930f906 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:57.600540 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a9c0e-6836-43b7-ae38-8a340930f906 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:57.613326 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a9c0e-6836-43b7-ae38-8a340930f906 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:57.621463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-643a9c0e-6836-43b7-ae38-8a340930f906 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:14:57.621463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:57.621463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:57.621463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:57.621463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:57.621463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:57.621463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:57.621463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:57.621463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:57.621463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:57.621463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:57.621463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:57.621463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:57.621463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:57.621463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:57.621463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:57.622523 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:57.622523 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:57.622523 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:57.622523 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:57.622523 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:57.622523 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:57.622523 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:57.622523 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:57.622523 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:57.622523 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:57.622523 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:57.622523 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:14:57.622523 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:57.622523 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:57.622523 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:57.622523 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:57.623524 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:57.623524 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:57.623524 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:57.623524 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:57.623524 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:57.623524 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:57.623524 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:57.623524 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:57.623524 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:57.623524 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:57.623524 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:14:57.623524 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:14:57.624509 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 199/408] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:14:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:57.625738 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fdd3b6-63d2-419b-91d8-5be939e9c3b0 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:57.626022 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fdd3b6-63d2-419b-91d8-5be939e9c3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:57.626298 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fdd3b6-63d2-419b-91d8-5be939e9c3b0 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:57.628332 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 200/409] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:57.634975 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fdd3b6-63d2-419b-91d8-5be939e9c3b0 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:57.635449 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc04c79-8b7a-4a67-8315-f830eb33a5e0 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:57.680246 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee3019-d0bc-4928-bdac-1f899860cf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:57.681487 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee3019-d0bc-4928-bdac-1f899860cf98 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:57.681924 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee3019-d0bc-4928-bdac-1f899860cf98 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:57.682424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee3019-d0bc-4928-bdac-1f899860cf98 None admin] PATH_INFO: `/v3/groups/01e533eb757846d9927b53ab53f7edf5` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:57.683870 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee3019-d0bc-4928-bdac-1f899860cf98 None admin] RBAC: Authorizing `identity:get_group(group_id=01e533eb757846d9927b53ab53f7edf5)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:57.685447 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee3019-d0bc-4928-bdac-1f899860cf98 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:57.687225 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 210/410] 10.222.0.24 () {56 vars in 1180 bytes} [Fri May 27 03:14:57 2022] GET /identity/v3/groups/01e533eb757846d9927b53ab53f7edf5 => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:57.688614 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57140d9-c53a-4b7c-8573-c5c04c753cd1 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:57.689032 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57140d9-c53a-4b7c-8573-c5c04c753cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:57.689445 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57140d9-c53a-4b7c-8573-c5c04c753cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:57.707272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-e57140d9-c53a-4b7c-8573-c5c04c753cd1 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:57.708805 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e57140d9-c53a-4b7c-8573-c5c04c753cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:57.767448 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fdd3b6-63d2-419b-91d8-5be939e9c3b0 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:57.785120 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 211/411] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:14:57.786252 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57140d9-c53a-4b7c-8573-c5c04c753cd1 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:57.816576 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:57.832407 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266632f1-899e-4f0a-9421-adc746fd5787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:57.833877 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266632f1-899e-4f0a-9421-adc746fd5787 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:57.834454 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266632f1-899e-4f0a-9421-adc746fd5787 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:57.834971 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266632f1-899e-4f0a-9421-adc746fd5787 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:57.836379 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266632f1-899e-4f0a-9421-adc746fd5787 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:57.837140 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266632f1-899e-4f0a-9421-adc746fd5787 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:57.838707 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266632f1-899e-4f0a-9421-adc746fd5787 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:57.850961 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 201/412] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:57.852226 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266632f1-899e-4f0a-9421-adc746fd5787 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:57.909214 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:57.926495 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57140d9-c53a-4b7c-8573-c5c04c753cd1 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:57.965844 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef87600-e9a9-450d-9c59-5be6169b69d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:57.966631 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef87600-e9a9-450d-9c59-5be6169b69d9 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:57.966939 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef87600-e9a9-450d-9c59-5be6169b69d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:57.967418 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef87600-e9a9-450d-9c59-5be6169b69d9 None admin] PATH_INFO: `/v3/projects/d95eae40782d41759de176dea214eb16` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:57.968846 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef87600-e9a9-450d-9c59-5be6169b69d9 None admin] RBAC: Authorizing `identity:get_project(project_id=d95eae40782d41759de176dea214eb16)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:57.970060 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef87600-e9a9-450d-9c59-5be6169b69d9 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:57.976541 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 212/413] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:57 2022] GET /identity/v3/projects/d95eae40782d41759de176dea214eb16 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:57.977341 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef87600-e9a9-450d-9c59-5be6169b69d9 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:57.992068 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266632f1-899e-4f0a-9421-adc746fd5787 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:58.038168 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef457c-d531-4826-b060-52f41a02828d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:58.039562 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef457c-d531-4826-b060-52f41a02828d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:58.040439 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef457c-d531-4826-b060-52f41a02828d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:58.041181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef457c-d531-4826-b060-52f41a02828d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:58.053758 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef457c-d531-4826-b060-52f41a02828d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:58.061820 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-fbef457c-d531-4826-b060-52f41a02828d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:58.061820 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:58.061820 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:58.061820 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:58.061820 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:58.061820 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:58.061820 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:58.061820 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:58.061820 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:58.061820 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:58.061820 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:58.061820 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:58.061820 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:58.061820 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:58.061820 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:58.061820 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:58.062726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:58.062726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:58.062726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:58.062726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:58.062726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:58.062726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:58.062726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:58.062726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:58.062726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:58.062726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:58.062726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:58.062726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:14:58.062726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:58.062726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:58.062726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:58.062726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:58.069206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:58.069206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:58.069206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:58.069206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:58.069206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:58.069206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:58.069206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:58.069206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:58.069206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:58.069206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:58.069206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:58.069206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:14:58.069206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 202/414] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:58.069206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef457c-d531-4826-b060-52f41a02828d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:58.089040 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:58.106046 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef87600-e9a9-450d-9c59-5be6169b69d9 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:58.143657 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c69c45-c4f0-49b6-9d2e-accc806a6cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:58.144521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c69c45-c4f0-49b6-9d2e-accc806a6cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:58.144803 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c69c45-c4f0-49b6-9d2e-accc806a6cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:58.145081 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c69c45-c4f0-49b6-9d2e-accc806a6cd0 None admin] PATH_INFO: `/v3/groups/6246a690b6ed4faf896eba90272c3265` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:58.146576 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c69c45-c4f0-49b6-9d2e-accc806a6cd0 None admin] RBAC: Authorizing `identity:get_group(group_id=6246a690b6ed4faf896eba90272c3265)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:58.147866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c69c45-c4f0-49b6-9d2e-accc806a6cd0 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:58.149918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 213/415] 10.222.0.24 () {56 vars in 1180 bytes} [Fri May 27 03:14:57 2022] GET /identity/v3/groups/6246a690b6ed4faf896eba90272c3265 => generated 229 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:58.150859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c69c45-c4f0-49b6-9d2e-accc806a6cd0 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:58.195918 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:58.199593 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef457c-d531-4826-b060-52f41a02828d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:58.237077 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37525269-da74-4c25-a913-086ee3a57bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:58.238049 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37525269-da74-4c25-a913-086ee3a57bc7 None admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:58.238487 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37525269-da74-4c25-a913-086ee3a57bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:58.238920 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37525269-da74-4c25-a913-086ee3a57bc7 None admin] PATH_INFO: `/v3/projects/d95eae40782d41759de176dea214eb16/groups/01e533eb757846d9927b53ab53f7edf5/roles/d7133ce917f0431883775551ab086d5d` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:58.241399 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37525269-da74-4c25-a913-086ee3a57bc7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d95eae40782d41759de176dea214eb16, group_id=01e533eb757846d9927b53ab53f7edf5, role_id=d7133ce917f0431883775551ab086d5d)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:58.242992 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37525269-da74-4c25-a913-086ee3a57bc7 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:58.277322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c69c45-c4f0-49b6-9d2e-accc806a6cd0 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:58.307312 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 203/416] 10.222.0.24 () {58 vars in 1442 bytes} [Fri May 27 03:14:58 2022] PUT /identity/v3/projects/d95eae40782d41759de176dea214eb16/groups/01e533eb757846d9927b53ab53f7edf5/roles/d7133ce917f0431883775551ab086d5d => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:14:58.309199 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37525269-da74-4c25-a913-086ee3a57bc7 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:58.328095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3433416-ac1a-49ec-835f-17667ee43a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:58.329170 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3433416-ac1a-49ec-835f-17667ee43a5d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:58.329712 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3433416-ac1a-49ec-835f-17667ee43a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:58.330210 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3433416-ac1a-49ec-835f-17667ee43a5d None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:58.331285 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3433416-ac1a-49ec-835f-17667ee43a5d None admin] RBAC: Adding query filter params (name=member) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:58.331951 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3433416-ac1a-49ec-835f-17667ee43a5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:58.336840 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3433416-ac1a-49ec-835f-17667ee43a5d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:58.364329 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 214/417] 10.222.0.24 () {56 vars in 1101 bytes} [Fri May 27 03:14:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:58.380669 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3433416-ac1a-49ec-835f-17667ee43a5d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:58.413975 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:58.501799 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37525269-da74-4c25-a913-086ee3a57bc7 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:58.539378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2ec9d-dff3-42f1-b789-ea942ba35d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:58.540548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2ec9d-dff3-42f1-b789-ea942ba35d97 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:58.541031 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2ec9d-dff3-42f1-b789-ea942ba35d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:58.541527 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2ec9d-dff3-42f1-b789-ea942ba35d97 None admin] PATH_INFO: `/v3/projects/4d3e8df9c2aa4b4588906f73830bd805` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:58.543017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2ec9d-dff3-42f1-b789-ea942ba35d97 None admin] RBAC: Authorizing `identity:get_project(project_id=4d3e8df9c2aa4b4588906f73830bd805)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:58.544640 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2ec9d-dff3-42f1-b789-ea942ba35d97 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:58.546866 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 204/418] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:58 2022] GET /identity/v3/projects/4d3e8df9c2aa4b4588906f73830bd805 => generated 344 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:58.549932 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3433416-ac1a-49ec-835f-17667ee43a5d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:58.552087 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2ec9d-dff3-42f1-b789-ea942ba35d97 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:58.588945 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:58.590266 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c63ed-1ecd-40ab-8f48-940d814c5e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:58.591050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c63ed-1ecd-40ab-8f48-940d814c5e76 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:58.591404 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c63ed-1ecd-40ab-8f48-940d814c5e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:58.591755 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c63ed-1ecd-40ab-8f48-940d814c5e76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:58.678027 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2ec9d-dff3-42f1-b789-ea942ba35d97 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:58.717575 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d159a-be4a-485b-b1dc-52789b398cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:58.718757 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d159a-be4a-485b-b1dc-52789b398cc4 None admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:58.719265 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d159a-be4a-485b-b1dc-52789b398cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:58.719855 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d159a-be4a-485b-b1dc-52789b398cc4 None admin] PATH_INFO: `/v3/projects/4d3e8df9c2aa4b4588906f73830bd805/groups/6246a690b6ed4faf896eba90272c3265/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:58.722545 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d159a-be4a-485b-b1dc-52789b398cc4 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d3e8df9c2aa4b4588906f73830bd805, group_id=6246a690b6ed4faf896eba90272c3265, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:58.724244 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d159a-be4a-485b-b1dc-52789b398cc4 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:58.731653 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c63ed-1ecd-40ab-8f48-940d814c5e76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:58.733191 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c63ed-1ecd-40ab-8f48-940d814c5e76 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:58.754490 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:58.809168 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 205/419] 10.222.0.24 () {58 vars in 1442 bytes} [Fri May 27 03:14:58 2022] PUT /identity/v3/projects/4d3e8df9c2aa4b4588906f73830bd805/groups/6246a690b6ed4faf896eba90272c3265/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:14:58.869565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 215/420] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:14:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:14:58.871935 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d159a-be4a-485b-b1dc-52789b398cc4 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:58.989151 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:58.994827 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff6374-1e5d-42ae-9f93-96f4fd9fc25f None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:58.995207 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff6374-1e5d-42ae-9f93-96f4fd9fc25f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:58.996035 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff6374-1e5d-42ae-9f93-96f4fd9fc25f None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:58.997737 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 216/421] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:59.003372 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96500a0f-daa2-4bdf-b71c-e85dc7de5361 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:59.003654 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96500a0f-daa2-4bdf-b71c-e85dc7de5361 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:59.003754 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d159a-be4a-485b-b1dc-52789b398cc4 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:59.003930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96500a0f-daa2-4bdf-b71c-e85dc7de5361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:59.021451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-96500a0f-daa2-4bdf-b71c-e85dc7de5361 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:59.022593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-96500a0f-daa2-4bdf-b71c-e85dc7de5361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:59.042019 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e3362-5361-4486-a87a-7f45af352214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:59.043005 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e3362-5361-4486-a87a-7f45af352214 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:59.043292 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e3362-5361-4486-a87a-7f45af352214 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:59.043569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e3362-5361-4486-a87a-7f45af352214 None admin] PATH_INFO: `/v3/projects/9dbc862abb4247f69fc1a65f462d65ce` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:59.044892 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e3362-5361-4486-a87a-7f45af352214 None admin] RBAC: Authorizing `identity:get_project(project_id=9dbc862abb4247f69fc1a65f462d65ce)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:59.046474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e3362-5361-4486-a87a-7f45af352214 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:59.048248 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 206/422] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:58 2022] GET /identity/v3/projects/9dbc862abb4247f69fc1a65f462d65ce => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:14:59.059754 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e3362-5361-4486-a87a-7f45af352214 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:59.132275 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 217/423] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:14:59.175904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96500a0f-daa2-4bdf-b71c-e85dc7de5361 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:59.217550 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e3362-5361-4486-a87a-7f45af352214 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:59.228647 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:59.254328 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135670c-49f4-45fe-bde9-09981013df54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:59.255716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135670c-49f4-45fe-bde9-09981013df54 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:59.255716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135670c-49f4-45fe-bde9-09981013df54 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:59.256168 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135670c-49f4-45fe-bde9-09981013df54 None admin] PATH_INFO: `/v3/groups/01e533eb757846d9927b53ab53f7edf5` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:59.257403 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6135670c-49f4-45fe-bde9-09981013df54 None admin] RBAC: Authorizing `identity:get_group(group_id=01e533eb757846d9927b53ab53f7edf5)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:59.259299 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6135670c-49f4-45fe-bde9-09981013df54 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:59.261155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 207/424] 10.222.0.24 () {56 vars in 1180 bytes} [Fri May 27 03:14:59 2022] GET /identity/v3/groups/01e533eb757846d9927b53ab53f7edf5 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:59.267272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135670c-49f4-45fe-bde9-09981013df54 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:59.293677 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96500a0f-daa2-4bdf-b71c-e85dc7de5361 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:59.300303 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:59.333614 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e33484-138a-4d92-8aac-cbf6d69ce352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:59.334661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e33484-138a-4d92-8aac-cbf6d69ce352 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:59.335397 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e33484-138a-4d92-8aac-cbf6d69ce352 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:59.335882 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e33484-138a-4d92-8aac-cbf6d69ce352 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:59.348017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e33484-138a-4d92-8aac-cbf6d69ce352 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:14:59.357374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-16e33484-138a-4d92-8aac-cbf6d69ce352 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:59.357374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:14:59.357374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:14:59.357374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:14:59.357374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:14:59.357374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:14:59.357374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:14:59.357374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:14:59.357374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:14:59.357374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:14:59.357374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:14:59.357374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:14:59.357374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:14:59.357374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:14:59.357374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:14:59.357374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:14:59.358295 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:14:59.358295 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:14:59.358295 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:14:59.358295 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:14:59.358295 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:14:59.358295 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:14:59.358295 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:14:59.358295 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:14:59.358295 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:14:59.358295 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:14:59.358295 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:14:59.358295 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:14:59.358295 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:14:59.358295 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:14:59.358295 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:14:59.358295 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:14:59.359110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:14:59.359110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:14:59.359110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:14:59.359110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:14:59.359110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:14:59.359110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:14:59.359110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:14:59.359110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:14:59.359110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:14:59.359110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:14:59.359110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:14:59.359110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:14:59.361934 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 218/425] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:14:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:14:59.370706 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e33484-138a-4d92-8aac-cbf6d69ce352 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:59.394304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135670c-49f4-45fe-bde9-09981013df54 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:59.431776 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf71ab4-a479-4df0-b378-e36adc7de1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:59.432320 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf71ab4-a479-4df0-b378-e36adc7de1fc None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:59.432603 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf71ab4-a479-4df0-b378-e36adc7de1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:59.432903 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf71ab4-a479-4df0-b378-e36adc7de1fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:59.433743 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf71ab4-a479-4df0-b378-e36adc7de1fc None admin] RBAC: Adding query filter params (group.id=01e533eb757846d9927b53ab53f7edf5, role.id=343337c6f25943a9ad412047b70ddefa, scope.project.id=9dbc862abb4247f69fc1a65f462d65ce) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:59.434077 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf71ab4-a479-4df0-b378-e36adc7de1fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:59.435660 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf71ab4-a479-4df0-b378-e36adc7de1fc None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:59.442285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 208/426] 10.222.0.24 () {56 vars in 1376 bytes} [Fri May 27 03:14:59 2022] GET /identity/v3/role_assignments?group.id=01e533eb757846d9927b53ab53f7edf5&scope.project.id=9dbc862abb4247f69fc1a65f462d65ce&role.id=343337c6f25943a9ad412047b70ddefa => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:59.495175 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e33484-138a-4d92-8aac-cbf6d69ce352 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:59.498521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:59.533313 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e6e2d-2e47-4742-97fb-7d8b2a9a4b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:59.533996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e6e2d-2e47-4742-97fb-7d8b2a9a4b4b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:59.534407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e6e2d-2e47-4742-97fb-7d8b2a9a4b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:59.534597 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e6e2d-2e47-4742-97fb-7d8b2a9a4b4b None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:59.535340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403e6e2d-2e47-4742-97fb-7d8b2a9a4b4b None admin] RBAC: Adding query filter params (name=member) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:14:59.536052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403e6e2d-2e47-4742-97fb-7d8b2a9a4b4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:59.537632 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403e6e2d-2e47-4742-97fb-7d8b2a9a4b4b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:59.547799 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 219/427] 10.222.0.24 () {56 vars in 1101 bytes} [Fri May 27 03:14:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:14:59.553637 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf71ab4-a479-4df0-b378-e36adc7de1fc None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:59.653999 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:59.668944 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf71ab4-a479-4df0-b378-e36adc7de1fc None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:59.708275 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afd9fee-6650-48aa-8a09-dcee09c6d689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:59.708952 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd9fee-6650-48aa-8a09-dcee09c6d689 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:59.709235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd9fee-6650-48aa-8a09-dcee09c6d689 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:59.709509 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd9fee-6650-48aa-8a09-dcee09c6d689 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:59.781807 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afd9fee-6650-48aa-8a09-dcee09c6d689 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:59.783327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afd9fee-6650-48aa-8a09-dcee09c6d689 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:59.875021 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:59.887851 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 209/428] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:14:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:14:59.892857 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4e2ed-c1dc-466b-99b9-35b1b8b20726 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:59.893147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4e2ed-c1dc-466b-99b9-35b1b8b20726 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:59.893423 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4e2ed-c1dc-466b-99b9-35b1b8b20726 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:59.894920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 220/429] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:59.897103 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afd9fee-6650-48aa-8a09-dcee09c6d689 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:59.900944 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92f1a2-5df6-4408-a0be-8f7901aee5be None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:59.901230 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92f1a2-5df6-4408-a0be-8f7901aee5be None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:59.901506 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c92f1a2-5df6-4408-a0be-8f7901aee5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:59.936499 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-0c92f1a2-5df6-4408-a0be-8f7901aee5be None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:59.937987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-0c92f1a2-5df6-4408-a0be-8f7901aee5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:00.019711 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afd9fee-6650-48aa-8a09-dcee09c6d689 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:00.025849 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 221/430] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:15:00.052793 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92f1a2-5df6-4408-a0be-8f7901aee5be None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:00.070561 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d22b2-7419-4121-8fb9-6758e361a03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:00.071796 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d22b2-7419-4121-8fb9-6758e361a03b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:00.072296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d22b2-7419-4121-8fb9-6758e361a03b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:00.072759 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7d22b2-7419-4121-8fb9-6758e361a03b None admin] PATH_INFO: `/v3/projects/d95eae40782d41759de176dea214eb16` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:00.074286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7d22b2-7419-4121-8fb9-6758e361a03b None admin] RBAC: Authorizing `identity:get_project(project_id=d95eae40782d41759de176dea214eb16)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:00.076405 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7d22b2-7419-4121-8fb9-6758e361a03b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:00.078451 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 210/431] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:14:59 2022] GET /identity/v3/projects/d95eae40782d41759de176dea214eb16 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:00.084297 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d22b2-7419-4121-8fb9-6758e361a03b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:00.158437 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:00.177645 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:00.185570 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c92f1a2-5df6-4408-a0be-8f7901aee5be None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:00.211486 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7d22b2-7419-4121-8fb9-6758e361a03b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:00.229136 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a1df3-9d58-4fda-a8fb-8a45420aa65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:00.230807 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a1df3-9d58-4fda-a8fb-8a45420aa65b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:00.231111 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a1df3-9d58-4fda-a8fb-8a45420aa65b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:00.231381 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a1df3-9d58-4fda-a8fb-8a45420aa65b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:00.245674 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a1df3-9d58-4fda-a8fb-8a45420aa65b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:00.248830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6cfc8-84f3-450a-9f7c-85e49ce67c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:00.249693 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6cfc8-84f3-450a-9f7c-85e49ce67c26 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:00.249970 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6cfc8-84f3-450a-9f7c-85e49ce67c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:00.250240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6cfc8-84f3-450a-9f7c-85e49ce67c26 None admin] PATH_INFO: `/v3/groups/01e533eb757846d9927b53ab53f7edf5` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:00.251438 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6cfc8-84f3-450a-9f7c-85e49ce67c26 None admin] RBAC: Authorizing `identity:get_group(group_id=01e533eb757846d9927b53ab53f7edf5)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:00.252936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d6cfc8-84f3-450a-9f7c-85e49ce67c26 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:00.254020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-140a1df3-9d58-4fda-a8fb-8a45420aa65b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:15:00.254020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:00.254020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:00.254020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:00.254020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:00.254020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:00.254020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:00.254020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:00.254020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:00.254020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:00.254020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:00.254020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:00.254020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:00.254020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:00.254020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:00.254020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:00.255160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:00.255160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:00.255160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:00.255160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:00.255160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:00.255160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:00.255160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:00.255160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:00.255160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:00.255160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:00.255160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:00.255160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:00.255160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:00.255160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:00.255160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:00.255160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:00.256232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:00.256232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:00.256232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:00.256232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:00.256232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:00.256232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:00.256232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:00.256232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:00.256232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:00.256232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:00.256232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:15:00.256232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:00.256828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 211/432] 10.222.0.24 () {56 vars in 1180 bytes} [Fri May 27 03:15:00 2022] GET /identity/v3/groups/01e533eb757846d9927b53ab53f7edf5 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:00.257218 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 222/433] 10.222.0.24 () {56 vars in 1114 bytes} [Fri May 27 03:15:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:00.265700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6cfc8-84f3-450a-9f7c-85e49ce67c26 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:00.267427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a1df3-9d58-4fda-a8fb-8a45420aa65b None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:00.400459 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a1df3-9d58-4fda-a8fb-8a45420aa65b None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:00.409542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d6cfc8-84f3-450a-9f7c-85e49ce67c26 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:00.436671 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:00.440317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcc754-7162-43a9-a578-5f486ba4d031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:00.441148 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcc754-7162-43a9-a578-5f486ba4d031 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:00.441414 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcc754-7162-43a9-a578-5f486ba4d031 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:00.441757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcc754-7162-43a9-a578-5f486ba4d031 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:00.442563 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bcc754-7162-43a9-a578-5f486ba4d031 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:00.442970 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bcc754-7162-43a9-a578-5f486ba4d031 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:00.444413 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bcc754-7162-43a9-a578-5f486ba4d031 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:00.450808 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973f626-a86c-4db7-a74c-c6563bcab493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:00.452015 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973f626-a86c-4db7-a74c-c6563bcab493 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:00.452491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973f626-a86c-4db7-a74c-c6563bcab493 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:00.452947 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973f626-a86c-4db7-a74c-c6563bcab493 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:00.453973 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a973f626-a86c-4db7-a74c-c6563bcab493 None admin] RBAC: Adding query filter params (group.id=01e533eb757846d9927b53ab53f7edf5, role.id=343337c6f25943a9ad412047b70ddefa, scope.project.id=d95eae40782d41759de176dea214eb16) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:00.454467 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a973f626-a86c-4db7-a74c-c6563bcab493 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:00.454816 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 223/434] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:00.456479 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a973f626-a86c-4db7-a74c-c6563bcab493 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:00.460700 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcc754-7162-43a9-a578-5f486ba4d031 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:00.464622 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 212/435] 10.222.0.24 () {56 vars in 1376 bytes} [Fri May 27 03:15:00 2022] GET /identity/v3/role_assignments?group.id=01e533eb757846d9927b53ab53f7edf5&scope.project.id=d95eae40782d41759de176dea214eb16&role.id=343337c6f25943a9ad412047b70ddefa => generated 611 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:00.482689 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:00.574613 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcc754-7162-43a9-a578-5f486ba4d031 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:00.612113 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ff7b0-e7ac-4cc8-89a3-6a85b2312294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:00.612780 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ff7b0-e7ac-4cc8-89a3-6a85b2312294 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:00.613060 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ff7b0-e7ac-4cc8-89a3-6a85b2312294 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:00.613335 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ff7b0-e7ac-4cc8-89a3-6a85b2312294 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:00.687014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ff7b0-e7ac-4cc8-89a3-6a85b2312294 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:00.688397 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ff7b0-e7ac-4cc8-89a3-6a85b2312294 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:00.707936 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:00.791031 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 224/436] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:15:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:15:00.796587 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973f626-a86c-4db7-a74c-c6563bcab493 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:00.899524 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973f626-a86c-4db7-a74c-c6563bcab493 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:00.932198 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc26a02-3821-4476-9f53-1a49987ebfed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:00.933603 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc26a02-3821-4476-9f53-1a49987ebfed None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:00.933879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc26a02-3821-4476-9f53-1a49987ebfed None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:00.934148 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc26a02-3821-4476-9f53-1a49987ebfed None admin] PATH_INFO: `/v3/projects/9dbc862abb4247f69fc1a65f462d65ce` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:00.935345 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc26a02-3821-4476-9f53-1a49987ebfed None admin] RBAC: Authorizing `identity:get_project(project_id=9dbc862abb4247f69fc1a65f462d65ce)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:00.936794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc26a02-3821-4476-9f53-1a49987ebfed None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:00.938681 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 213/437] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:15:00 2022] GET /identity/v3/projects/9dbc862abb4247f69fc1a65f462d65ce => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:00.943911 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ff7b0-e7ac-4cc8-89a3-6a85b2312294 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:01.002546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435f945-a4ba-417a-bafe-1e32c620127a None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:01.002867 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435f945-a4ba-417a-bafe-1e32c620127a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:01.003141 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435f945-a4ba-417a-bafe-1e32c620127a None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:01.004577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 214/438] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:15:01.009421 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9122790-70e6-4f2a-83e2-c5a98ee4df73 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:01.009705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9122790-70e6-4f2a-83e2-c5a98ee4df73 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:01.009982 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9122790-70e6-4f2a-83e2-c5a98ee4df73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:01.027333 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e9122790-70e6-4f2a-83e2-c5a98ee4df73 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:01.028583 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e9122790-70e6-4f2a-83e2-c5a98ee4df73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:01.044490 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ff7b0-e7ac-4cc8-89a3-6a85b2312294 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:01.061494 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:01.081881 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0942aa-0fb5-4243-b2d7-712db1dc1050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:01.082947 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0942aa-0fb5-4243-b2d7-712db1dc1050 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:01.083369 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0942aa-0fb5-4243-b2d7-712db1dc1050 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:01.083808 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0942aa-0fb5-4243-b2d7-712db1dc1050 None admin] PATH_INFO: `/v3/groups/01e533eb757846d9927b53ab53f7edf5` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:01.085881 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0942aa-0fb5-4243-b2d7-712db1dc1050 None admin] RBAC: Authorizing `identity:get_group(group_id=01e533eb757846d9927b53ab53f7edf5)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:01.087354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0942aa-0fb5-4243-b2d7-712db1dc1050 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:01.087921 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:01.089161 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 225/439] 10.222.0.24 () {56 vars in 1180 bytes} [Fri May 27 03:15:00 2022] GET /identity/v3/groups/01e533eb757846d9927b53ab53f7edf5 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:01.095061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0942aa-0fb5-4243-b2d7-712db1dc1050 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:01.106018 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 215/440] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:15:01.128893 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9122790-70e6-4f2a-83e2-c5a98ee4df73 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:01.204956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0942aa-0fb5-4243-b2d7-712db1dc1050 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:01.230906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9122790-70e6-4f2a-83e2-c5a98ee4df73 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:01.241376 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6c61f-962a-47d7-8595-d25c12f88bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:01.242143 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6c61f-962a-47d7-8595-d25c12f88bac None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:01.242504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6c61f-962a-47d7-8595-d25c12f88bac None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:01.242868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6c61f-962a-47d7-8595-d25c12f88bac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:01.243698 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf6c61f-962a-47d7-8595-d25c12f88bac None admin] RBAC: Adding query filter params (group.id=01e533eb757846d9927b53ab53f7edf5, role.id=d7133ce917f0431883775551ab086d5d, scope.project.id=9dbc862abb4247f69fc1a65f462d65ce) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:01.244200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf6c61f-962a-47d7-8595-d25c12f88bac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:01.246274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf6c61f-962a-47d7-8595-d25c12f88bac None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:01.261759 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 226/441] 10.222.0.24 () {56 vars in 1376 bytes} [Fri May 27 03:15:01 2022] GET /identity/v3/role_assignments?group.id=01e533eb757846d9927b53ab53f7edf5&scope.project.id=9dbc862abb4247f69fc1a65f462d65ce&role.id=d7133ce917f0431883775551ab086d5d => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:15:01.265538 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95685f2b-4e16-47a3-bfea-26aa899610ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:01.266385 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95685f2b-4e16-47a3-bfea-26aa899610ec None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:01.266670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95685f2b-4e16-47a3-bfea-26aa899610ec None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:01.266987 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95685f2b-4e16-47a3-bfea-26aa899610ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:01.279541 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95685f2b-4e16-47a3-bfea-26aa899610ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:01.287966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-95685f2b-4e16-47a3-bfea-26aa899610ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:15:01.287966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:01.287966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:01.287966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:01.287966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:01.287966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:01.287966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:01.287966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:01.287966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:01.287966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:01.287966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:01.287966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:01.287966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:01.287966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:01.287966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:01.287966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:01.289002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:01.289002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:01.289002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:01.289002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:01.289002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:01.289002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:01.289002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:01.289002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:01.289002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:01.289002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:01.289002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:01.289002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:01.289002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:01.289002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:01.289002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:01.289002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:01.289775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:01.289775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:01.289775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:01.289775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:01.289775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:01.289775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:01.289775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:01.289775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:01.289775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:01.289775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:01.289775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:15:01.289775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:01.291115 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 216/442] 10.222.0.24 () {56 vars in 1114 bytes} [Fri May 27 03:15:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:15:01.295775 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6c61f-962a-47d7-8595-d25c12f88bac None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:01.352660 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:01.406865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6c61f-962a-47d7-8595-d25c12f88bac None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:01.447277 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7cc905-0f3d-431a-970f-715474c3f27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:01.447963 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7cc905-0f3d-431a-970f-715474c3f27d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:01.448251 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7cc905-0f3d-431a-970f-715474c3f27d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:01.448563 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7cc905-0f3d-431a-970f-715474c3f27d None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:01.449333 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7cc905-0f3d-431a-970f-715474c3f27d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:01.449639 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7cc905-0f3d-431a-970f-715474c3f27d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:01.451403 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7cc905-0f3d-431a-970f-715474c3f27d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:01.461541 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 227/443] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:15:01.467305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95685f2b-4e16-47a3-bfea-26aa899610ec None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:01.520075 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:01.580815 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95685f2b-4e16-47a3-bfea-26aa899610ec None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:01.616642 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114b32e-f3d6-4d6d-a252-6bf396b238c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:01.617306 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114b32e-f3d6-4d6d-a252-6bf396b238c3 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:01.617592 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114b32e-f3d6-4d6d-a252-6bf396b238c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:01.617877 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114b32e-f3d6-4d6d-a252-6bf396b238c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:01.627472 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ce76b-2f6a-4f51-b063-23f073887d08 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:01.627766 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ce76b-2f6a-4f51-b063-23f073887d08 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:01.628045 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ce76b-2f6a-4f51-b063-23f073887d08 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:01.629766 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 228/444] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:15:01.634596 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a0721-cd32-457a-8784-334d4a36acaa None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:01.634914 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a0721-cd32-457a-8784-334d4a36acaa None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:01.635199 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84a0721-cd32-457a-8784-334d4a36acaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:01.654134 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-f84a0721-cd32-457a-8784-334d4a36acaa None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:01.655468 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f84a0721-cd32-457a-8784-334d4a36acaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:01.698110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114b32e-f3d6-4d6d-a252-6bf396b238c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:01.699636 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114b32e-f3d6-4d6d-a252-6bf396b238c3 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:01.743613 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 229/445] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:15:01.752913 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:01.767341 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a0721-cd32-457a-8784-334d4a36acaa None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:01.817013 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 217/446] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:15:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:15:01.817371 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:01.822718 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114b32e-f3d6-4d6d-a252-6bf396b238c3 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:01.879282 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84a0721-cd32-457a-8784-334d4a36acaa None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:01.912870 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f90f7-ee73-4494-85c6-6eba3f73cc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:01.913660 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f90f7-ee73-4494-85c6-6eba3f73cc1d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:01.913944 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f90f7-ee73-4494-85c6-6eba3f73cc1d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:01.914221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f90f7-ee73-4494-85c6-6eba3f73cc1d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:01.924259 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f90f7-ee73-4494-85c6-6eba3f73cc1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:01.926772 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114b32e-f3d6-4d6d-a252-6bf396b238c3 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:01.932201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-001f90f7-ee73-4494-85c6-6eba3f73cc1d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:15:01.932201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:01.932201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:01.932201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:01.932201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:01.932201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:01.932201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:01.932201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:01.932201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:01.932201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:01.932201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:01.932201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:01.932201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:01.932201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:01.932201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:01.932201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:01.933346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:01.933346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:01.933346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:01.933346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:01.933346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:01.933346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:01.933346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:01.933346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:01.933346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:01.933346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:01.933346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:01.933346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:01.933346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:01.933346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:01.933346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:01.933346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:01.934167 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:01.934167 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:01.934167 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:01.934167 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:01.934167 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:01.934167 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:01.934167 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:01.934167 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:01.934167 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:01.934167 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:01.934167 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:15:01.934167 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:01.934167 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 230/447] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:15:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:15:01.939501 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f90f7-ee73-4494-85c6-6eba3f73cc1d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:01.967020 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76122a42-2a35-40cd-98a2-395380c04ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:01.967905 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76122a42-2a35-40cd-98a2-395380c04ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:01.968181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76122a42-2a35-40cd-98a2-395380c04ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:01.968451 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76122a42-2a35-40cd-98a2-395380c04ad2 None admin] PATH_INFO: `/v3/projects/d95eae40782d41759de176dea214eb16` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:01.969723 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76122a42-2a35-40cd-98a2-395380c04ad2 None admin] RBAC: Authorizing `identity:get_project(project_id=d95eae40782d41759de176dea214eb16)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:01.971167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76122a42-2a35-40cd-98a2-395380c04ad2 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:01.972993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 218/448] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:15:01 2022] GET /identity/v3/projects/d95eae40782d41759de176dea214eb16 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:15:01.978157 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76122a42-2a35-40cd-98a2-395380c04ad2 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:01.992741 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:02.044155 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f90f7-ee73-4494-85c6-6eba3f73cc1d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:02.045673 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:02.077097 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f67af-b836-4aa9-9a74-33d52a10052a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:02.077888 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f67af-b836-4aa9-9a74-33d52a10052a None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:02.078070 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f67af-b836-4aa9-9a74-33d52a10052a None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:02.080654 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f67af-b836-4aa9-9a74-33d52a10052a None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:02.080791 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76122a42-2a35-40cd-98a2-395380c04ad2 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:02.080884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f67af-b836-4aa9-9a74-33d52a10052a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:02.080884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f67af-b836-4aa9-9a74-33d52a10052a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:02.081102 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f67af-b836-4aa9-9a74-33d52a10052a None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:02.090113 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 231/449] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:15:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:15:02.095820 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f67af-b836-4aa9-9a74-33d52a10052a None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:02.114341 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b2683-6db9-43cf-9589-829b1d3a6215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:02.115224 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b2683-6db9-43cf-9589-829b1d3a6215 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:02.115509 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b2683-6db9-43cf-9589-829b1d3a6215 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:02.115782 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b2683-6db9-43cf-9589-829b1d3a6215 None admin] PATH_INFO: `/v3/groups/01e533eb757846d9927b53ab53f7edf5` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:02.116959 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b2683-6db9-43cf-9589-829b1d3a6215 None admin] RBAC: Authorizing `identity:get_group(group_id=01e533eb757846d9927b53ab53f7edf5)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:02.118399 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b2683-6db9-43cf-9589-829b1d3a6215 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:02.120040 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 219/450] 10.222.0.24 () {56 vars in 1180 bytes} [Fri May 27 03:15:01 2022] GET /identity/v3/groups/01e533eb757846d9927b53ab53f7edf5 => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:15:02.125612 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b2683-6db9-43cf-9589-829b1d3a6215 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:02.196352 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f67af-b836-4aa9-9a74-33d52a10052a None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:02.204533 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:02.222119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b2683-6db9-43cf-9589-829b1d3a6215 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:02.230698 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346aeca-e94a-45f1-9566-c57c82607de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:02.231375 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346aeca-e94a-45f1-9566-c57c82607de1 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:02.231656 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346aeca-e94a-45f1-9566-c57c82607de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:02.231930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346aeca-e94a-45f1-9566-c57c82607de1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:02.256183 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bad9ea-dab1-40a9-a205-79158d8d0e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:02.256820 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bad9ea-dab1-40a9-a205-79158d8d0e25 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:02.257102 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bad9ea-dab1-40a9-a205-79158d8d0e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:02.257370 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bad9ea-dab1-40a9-a205-79158d8d0e25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:02.258189 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bad9ea-dab1-40a9-a205-79158d8d0e25 None admin] RBAC: Adding query filter params (group.id=01e533eb757846d9927b53ab53f7edf5, role.id=d7133ce917f0431883775551ab086d5d, scope.project.id=d95eae40782d41759de176dea214eb16) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:02.258495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bad9ea-dab1-40a9-a205-79158d8d0e25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:02.259961 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bad9ea-dab1-40a9-a205-79158d8d0e25 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:02.261500 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:02.265718 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 220/451] 10.222.0.24 () {56 vars in 1376 bytes} [Fri May 27 03:15:02 2022] GET /identity/v3/role_assignments?group.id=01e533eb757846d9927b53ab53f7edf5&scope.project.id=d95eae40782d41759de176dea214eb16&role.id=d7133ce917f0431883775551ab086d5d => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:15:02.302198 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346aeca-e94a-45f1-9566-c57c82607de1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:02.303626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346aeca-e94a-45f1-9566-c57c82607de1 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:02.408659 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 232/452] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:15:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:15:02.414435 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bad9ea-dab1-40a9-a205-79158d8d0e25 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:02.515525 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bad9ea-dab1-40a9-a205-79158d8d0e25 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:02.548655 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de78116-b624-48c5-9355-f9ded283dfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:02.549517 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de78116-b624-48c5-9355-f9ded283dfbe None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:02.549799 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de78116-b624-48c5-9355-f9ded283dfbe None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:02.550072 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de78116-b624-48c5-9355-f9ded283dfbe None admin] PATH_INFO: `/v3/projects/4d3e8df9c2aa4b4588906f73830bd805` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:02.551293 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de78116-b624-48c5-9355-f9ded283dfbe None admin] RBAC: Authorizing `identity:get_project(project_id=4d3e8df9c2aa4b4588906f73830bd805)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:02.552768 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de78116-b624-48c5-9355-f9ded283dfbe None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:02.554398 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 221/453] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:15:02 2022] GET /identity/v3/projects/4d3e8df9c2aa4b4588906f73830bd805 => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:02.560190 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346aeca-e94a-45f1-9566-c57c82607de1 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:02.622791 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:02.660314 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346aeca-e94a-45f1-9566-c57c82607de1 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:02.693404 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d028c4-94e2-421b-9789-c663615ef965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:02.694250 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d028c4-94e2-421b-9789-c663615ef965 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:02.694527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d028c4-94e2-421b-9789-c663615ef965 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:02.694797 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d028c4-94e2-421b-9789-c663615ef965 None admin] PATH_INFO: `/v3/groups/6246a690b6ed4faf896eba90272c3265` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:02.695969 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d028c4-94e2-421b-9789-c663615ef965 None admin] RBAC: Authorizing `identity:get_group(group_id=6246a690b6ed4faf896eba90272c3265)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:02.697386 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d028c4-94e2-421b-9789-c663615ef965 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:02.699548 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 233/454] 10.222.0.24 () {56 vars in 1180 bytes} [Fri May 27 03:15:02 2022] GET /identity/v3/groups/6246a690b6ed4faf896eba90272c3265 => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:02.705559 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de78116-b624-48c5-9355-f9ded283dfbe None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:02.721646 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:02.801282 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de78116-b624-48c5-9355-f9ded283dfbe None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:02.834055 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35530b1-6d62-424f-8cea-6a9052e05b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:02.834652 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35530b1-6d62-424f-8cea-6a9052e05b0f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:02.834957 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35530b1-6d62-424f-8cea-6a9052e05b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:02.835253 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35530b1-6d62-424f-8cea-6a9052e05b0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:02.836065 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35530b1-6d62-424f-8cea-6a9052e05b0f None admin] RBAC: Adding query filter params (group.id=6246a690b6ed4faf896eba90272c3265, role.id=e5a908bd3ed5445b82a6053246533b88, scope.project.id=4d3e8df9c2aa4b4588906f73830bd805) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:02.836373 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35530b1-6d62-424f-8cea-6a9052e05b0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:02.837806 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35530b1-6d62-424f-8cea-6a9052e05b0f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:02.843392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 222/455] 10.222.0.24 () {56 vars in 1376 bytes} [Fri May 27 03:15:02 2022] GET /identity/v3/role_assignments?group.id=6246a690b6ed4faf896eba90272c3265&scope.project.id=4d3e8df9c2aa4b4588906f73830bd805&role.id=e5a908bd3ed5445b82a6053246533b88 => generated 611 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:05.466318 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5dac9-14f1-4451-a445-3e576a76d4d4 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:05.466801 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5dac9-14f1-4451-a445-3e576a76d4d4 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:05.466801 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e5dac9-14f1-4451-a445-3e576a76d4d4 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:05.468172 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 234/456] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:05.478156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c7bbf-6433-4be3-8fe7-2531767d9acc None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:05.479006 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c7bbf-6433-4be3-8fe7-2531767d9acc None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:05.479952 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c7bbf-6433-4be3-8fe7-2531767d9acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:05.506680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-287c7bbf-6433-4be3-8fe7-2531767d9acc None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:05.508115 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-287c7bbf-6433-4be3-8fe7-2531767d9acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:05.606016 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 223/457] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:15:05.652494 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e5dac9-14f1-4451-a445-3e576a76d4d4 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:05.788458 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e5dac9-14f1-4451-a445-3e576a76d4d4 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:05.828495 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:05.832339 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207610e8-0334-41cc-bb24-5ffc50b5d5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:05.833141 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207610e8-0334-41cc-bb24-5ffc50b5d5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:05.833427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207610e8-0334-41cc-bb24-5ffc50b5d5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:05.833705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207610e8-0334-41cc-bb24-5ffc50b5d5f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:05.849779 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207610e8-0334-41cc-bb24-5ffc50b5d5f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:05.851280 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207610e8-0334-41cc-bb24-5ffc50b5d5f3 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:05.865013 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e64de-644d-40cd-b431-af7795d7a129 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:05.865631 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e64de-644d-40cd-b431-af7795d7a129 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:05.866084 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e64de-644d-40cd-b431-af7795d7a129 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:05.869052 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 224/458] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:05.872838 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-207610e8-0334-41cc-bb24-5ffc50b5d5f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:05.872838 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:05.872838 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:05.872838 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:05.872838 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:05.872838 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:05.872838 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:05.872838 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:05.872838 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:05.872838 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:05.872838 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:05.872838 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:05.872838 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:05.872838 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:05.872838 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:05.873738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:05.873738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:05.873738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:05.873738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:05.873738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:05.873738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:05.873738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:05.873738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:05.873738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:05.873738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:05.873738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:05.873738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:05.873738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:05.873738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:05.873738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:05.873738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:05.873738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:05.874626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:05.874626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:05.874626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:05.874626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:05.876003 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196cb8cd-fe61-47f5-9c55-086bf634a589 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:05.876440 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196cb8cd-fe61-47f5-9c55-086bf634a589 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:05.876854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196cb8cd-fe61-47f5-9c55-086bf634a589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:05.878817 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 235/459] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:05.879659 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207610e8-0334-41cc-bb24-5ffc50b5d5f3 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:05.895304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-196cb8cd-fe61-47f5-9c55-086bf634a589 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:05.896872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-196cb8cd-fe61-47f5-9c55-086bf634a589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:05.905440 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:05.979289 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 225/460] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:15:05.981162 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a716aa-d160-4cfb-bed3-84bd140c4498 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:05.981703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a716aa-d160-4cfb-bed3-84bd140c4498 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:05.982173 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a716aa-d160-4cfb-bed3-84bd140c4498 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:05.989634 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 226/461] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:05.995725 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10c123-eb6e-4969-a908-1c35111487b7 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:05.996303 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10c123-eb6e-4969-a908-1c35111487b7 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:05.996778 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10c123-eb6e-4969-a908-1c35111487b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:06.009653 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207610e8-0334-41cc-bb24-5ffc50b5d5f3 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:06.024474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-1c10c123-eb6e-4969-a908-1c35111487b7 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:06.026588 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1c10c123-eb6e-4969-a908-1c35111487b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:06.051337 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb8f65-1f0b-43f2-a054-8cf2a4b232c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:06.052717 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb8f65-1f0b-43f2-a054-8cf2a4b232c6 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:06.053453 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb8f65-1f0b-43f2-a054-8cf2a4b232c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:06.053945 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb8f65-1f0b-43f2-a054-8cf2a4b232c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:06.055078 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb8f65-1f0b-43f2-a054-8cf2a4b232c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:06.055830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb8f65-1f0b-43f2-a054-8cf2a4b232c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:06.058297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb8f65-1f0b-43f2-a054-8cf2a4b232c6 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:06.074859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 236/462] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:06.076579 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb8f65-1f0b-43f2-a054-8cf2a4b232c6 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:06.114953 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 227/463] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:15:06.116153 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10c123-eb6e-4969-a908-1c35111487b7 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:06.138177 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:06.189892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb8f65-1f0b-43f2-a054-8cf2a4b232c6 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:06.200392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:06.231848 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3576b3a-2e46-4173-b938-7a6623730263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:06.232911 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3576b3a-2e46-4173-b938-7a6623730263 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:06.233190 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3576b3a-2e46-4173-b938-7a6623730263 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:06.233459 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3576b3a-2e46-4173-b938-7a6623730263 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:06.234054 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c10c123-eb6e-4969-a908-1c35111487b7 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:06.246287 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3576b3a-2e46-4173-b938-7a6623730263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:06.248195 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3576b3a-2e46-4173-b938-7a6623730263 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:06.267993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d592c-def4-482d-b078-60ef495f2bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:06.268659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d592c-def4-482d-b078-60ef495f2bfe None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:06.268943 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d592c-def4-482d-b078-60ef495f2bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:06.269219 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d592c-def4-482d-b078-60ef495f2bfe None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:06.269565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-a3576b3a-2e46-4173-b938-7a6623730263 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:06.269565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:06.269565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:06.269565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:06.269565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:06.269565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:06.269565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:06.269565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:06.269565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:06.269565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:06.269565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:06.269565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:06.269565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:06.269565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:06.269565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:06.270377 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:06.270377 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:06.270377 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:06.270377 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:06.270377 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:06.270377 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:06.270377 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:06.270377 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:06.270377 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:06.270377 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:06.270377 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:06.270377 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:06.270377 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:06.270377 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:06.270377 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:06.270377 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:06.270377 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:06.276511 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:06.276511 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:06.276511 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:06.276511 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:06.276721 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754d592c-def4-482d-b078-60ef495f2bfe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:06.276802 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 237/464] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:06.276870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754d592c-def4-482d-b078-60ef495f2bfe None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:06.276935 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4381e9-861b-4ec4-8226-2d0b10f7a840 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:06.276935 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4381e9-861b-4ec4-8226-2d0b10f7a840 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:06.276935 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4381e9-861b-4ec4-8226-2d0b10f7a840 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:06.276935 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 238/465] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:06.276935 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4381e9-861b-4ec4-8226-2d0b10f7a840 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:06.372328 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:06.378814 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4381e9-861b-4ec4-8226-2d0b10f7a840 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:06.395226 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 228/466] 10.222.0.24 () {60 vars in 1132 bytes} [Fri May 27 03:15:06 2022] POST /identity/v3/users => generated 246 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:15:06.396061 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d592c-def4-482d-b078-60ef495f2bfe None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:06.419428 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879b71b7-0b7e-4020-90e4-c90c3d48b0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:06.420246 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b71b7-0b7e-4020-90e4-c90c3d48b0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:06.420530 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b71b7-0b7e-4020-90e4-c90c3d48b0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:06.420804 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b71b7-0b7e-4020-90e4-c90c3d48b0f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:06.433514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879b71b7-0b7e-4020-90e4-c90c3d48b0f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:06.435038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879b71b7-0b7e-4020-90e4-c90c3d48b0f7 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:06.455120 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-879b71b7-0b7e-4020-90e4-c90c3d48b0f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:06.455120 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:06.455120 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:06.455120 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:06.455120 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:06.455120 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:06.455120 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:06.455120 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:06.455120 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:06.455120 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:06.455120 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:06.455120 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:06.455120 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:06.455120 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:06.455120 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:06.456036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:06.456036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:06.456036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:06.456036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:06.456036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:06.456036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:06.456036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:06.456036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:06.456036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:06.456036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:06.456036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:06.456036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:06.456036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:06.456036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:06.456036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:06.456036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:06.456036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:06.456936 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:06.456936 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:06.456936 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:06.456936 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:06.456936 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 239/467] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:06.457855 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572d281-14cd-4e43-8977-d9a3b70a044a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:06.458190 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572d281-14cd-4e43-8977-d9a3b70a044a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:06.458470 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572d281-14cd-4e43-8977-d9a3b70a044a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:06.476716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-5572d281-14cd-4e43-8977-d9a3b70a044a None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:06.477933 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5572d281-14cd-4e43-8977-d9a3b70a044a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:06.541317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d592c-def4-482d-b078-60ef495f2bfe None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:06.558006 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 240/468] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:15:06.559086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572d281-14cd-4e43-8977-d9a3b70a044a None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:06.571109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:06.586695 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83428241-fd21-482b-812c-4d308da7bf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:06.587738 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83428241-fd21-482b-812c-4d308da7bf48 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:06.593916 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83428241-fd21-482b-812c-4d308da7bf48 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:06.593916 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83428241-fd21-482b-812c-4d308da7bf48 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:06.593916 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83428241-fd21-482b-812c-4d308da7bf48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:06.593916 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83428241-fd21-482b-812c-4d308da7bf48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:06.593916 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83428241-fd21-482b-812c-4d308da7bf48 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:06.605461 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 229/469] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:06.606162 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83428241-fd21-482b-812c-4d308da7bf48 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:06.674530 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:06.707473 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572d281-14cd-4e43-8977-d9a3b70a044a None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:06.735416 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83428241-fd21-482b-812c-4d308da7bf48 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:06.755171 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac076c0c-6c69-4d41-ac7c-b303ab7900d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:06.755839 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac076c0c-6c69-4d41-ac7c-b303ab7900d3 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:06.756122 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac076c0c-6c69-4d41-ac7c-b303ab7900d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:06.756916 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac076c0c-6c69-4d41-ac7c-b303ab7900d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:06.757653 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac076c0c-6c69-4d41-ac7c-b303ab7900d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:06.758302 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac076c0c-6c69-4d41-ac7c-b303ab7900d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:06.760458 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac076c0c-6c69-4d41-ac7c-b303ab7900d3 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:06.770956 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63725c-ac6e-4c56-bd55-3914c523a897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:06.771754 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63725c-ac6e-4c56-bd55-3914c523a897 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:06.772039 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63725c-ac6e-4c56-bd55-3914c523a897 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:06.772315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63725c-ac6e-4c56-bd55-3914c523a897 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:06.775555 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 241/470] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:06.777139 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac076c0c-6c69-4d41-ac7c-b303ab7900d3 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:06.783545 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a63725c-ac6e-4c56-bd55-3914c523a897 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:06.785015 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a63725c-ac6e-4c56-bd55-3914c523a897 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:06.804057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-7a63725c-ac6e-4c56-bd55-3914c523a897 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:06.804057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:06.804057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:06.804057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:06.804057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:06.804057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:06.804057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:06.804057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:06.804057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:06.804057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:06.804057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:06.804057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:06.804057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:06.804057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:06.804057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:06.805136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:06.805136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:06.805136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:06.805136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:06.805136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:06.805136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:06.805136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:06.805136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:06.805136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:06.805136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:06.805136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:06.805136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:06.805136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:06.805136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:06.805136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:06.805136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:06.805136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:06.806176 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:06.806176 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:06.806176 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:06.806176 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:06.806176 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 230/471] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:06.806176 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63725c-ac6e-4c56-bd55-3914c523a897 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:06.850791 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:06.898135 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac076c0c-6c69-4d41-ac7c-b303ab7900d3 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:06.922895 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63725c-ac6e-4c56-bd55-3914c523a897 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:06.939439 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb10973-41a1-4ad7-ac67-4a33334cd214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:06.940189 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb10973-41a1-4ad7-ac67-4a33334cd214 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:06.940473 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb10973-41a1-4ad7-ac67-4a33334cd214 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:06.940921 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb10973-41a1-4ad7-ac67-4a33334cd214 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:06.941738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb10973-41a1-4ad7-ac67-4a33334cd214 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:06.943540 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb10973-41a1-4ad7-ac67-4a33334cd214 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:06.961373 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2739d8ff-8eb1-4f60-bc24-dabaafcd6dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:06.962046 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739d8ff-8eb1-4f60-bc24-dabaafcd6dbd None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:06.962324 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739d8ff-8eb1-4f60-bc24-dabaafcd6dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:06.962595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739d8ff-8eb1-4f60-bc24-dabaafcd6dbd None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:06.963338 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2739d8ff-8eb1-4f60-bc24-dabaafcd6dbd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:06.964940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2739d8ff-8eb1-4f60-bc24-dabaafcd6dbd None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:07.015612 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:07.119043 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 231/472] 10.222.0.24 () {60 vars in 1132 bytes} [Fri May 27 03:15:06 2022] POST /identity/v3/users => generated 248 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:15:07.119750 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2739d8ff-8eb1-4f60-bc24-dabaafcd6dbd None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:07.127155 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 242/473] 10.222.0.24 () {60 vars in 1132 bytes} [Fri May 27 03:15:06 2022] POST /identity/v3/users => generated 248 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:15:07.128867 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725a1ff-9ad5-4fab-b247-d59ce9c6028a None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:07.129154 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725a1ff-9ad5-4fab-b247-d59ce9c6028a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:07.129804 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725a1ff-9ad5-4fab-b247-d59ce9c6028a None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:07.132275 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 243/474] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:07.144842 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245be8e-3523-4abb-85c1-25d0d0537482 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:07.145155 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245be8e-3523-4abb-85c1-25d0d0537482 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:07.145456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245be8e-3523-4abb-85c1-25d0d0537482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:07.179114 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-5245be8e-3523-4abb-85c1-25d0d0537482 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:07.180354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5245be8e-3523-4abb-85c1-25d0d0537482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:07.256508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:07.270283 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2739d8ff-8eb1-4f60-bc24-dabaafcd6dbd None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:07.326991 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c77c4-6faa-4e9b-8be4-7d168b7e3da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:07.329760 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c77c4-6faa-4e9b-8be4-7d168b7e3da2 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:07.330494 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c77c4-6faa-4e9b-8be4-7d168b7e3da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:07.332103 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c77c4-6faa-4e9b-8be4-7d168b7e3da2 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:07.333809 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c77c4-6faa-4e9b-8be4-7d168b7e3da2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:07.334661 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c77c4-6faa-4e9b-8be4-7d168b7e3da2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:07.338352 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c77c4-6faa-4e9b-8be4-7d168b7e3da2 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:07.354500 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 244/475] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:15:07.359585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 232/476] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:07.362376 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245be8e-3523-4abb-85c1-25d0d0537482 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:07.380361 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c77c4-6faa-4e9b-8be4-7d168b7e3da2 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:07.479891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245be8e-3523-4abb-85c1-25d0d0537482 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:07.497359 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c77c4-6faa-4e9b-8be4-7d168b7e3da2 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:07.515304 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:07.522683 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ff460-7cce-46f7-a541-d10a564862aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:07.524702 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ff460-7cce-46f7-a541-d10a564862aa None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:07.524702 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ff460-7cce-46f7-a541-d10a564862aa None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:07.524702 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ff460-7cce-46f7-a541-d10a564862aa None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:07.525307 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38ff460-7cce-46f7-a541-d10a564862aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:07.527374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38ff460-7cce-46f7-a541-d10a564862aa None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:07.537726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431a0b7-5717-4488-ab89-a2d85710d514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:07.538593 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431a0b7-5717-4488-ab89-a2d85710d514 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:07.538862 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431a0b7-5717-4488-ab89-a2d85710d514 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:07.539209 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431a0b7-5717-4488-ab89-a2d85710d514 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:07.540025 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8431a0b7-5717-4488-ab89-a2d85710d514 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:07.541447 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8431a0b7-5717-4488-ab89-a2d85710d514 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:07.595409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.sql.core [None req-8431a0b7-5717-4488-ab89-a2d85710d514 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 27 03:15:07.595409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 27 03:15:07.595409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [parameters: {'id': '15550dc6281a4359a56a9b9d3ae2e126', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 27 03:15:07.595409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80482) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 27 03:15:07.606706 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 245/477] 10.222.0.24 () {60 vars in 1132 bytes} [Fri May 27 03:15:07 2022] POST /identity/v3/users => generated 249 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:15:07.614466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-8431a0b7-5717-4488-ab89-a2d85710d514 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 27 03:15:07.614466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:07.614466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 03:15:07.614466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 03:15:07.614466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 03:15:07.614466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 03:15:07.614466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 03:15:07.614466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application result = self._query(query) May 27 03:15:07.614466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 03:15:07.614466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application conn.query(q) May 27 03:15:07.614466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 03:15:07.614466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 03:15:07.614466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 03:15:07.614466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application result.read() May 27 03:15:07.614466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 03:15:07.615372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 03:15:07.615372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 03:15:07.615372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 03:15:07.615372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 03:15:07.615372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 03:15:07.615372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 03:15:07.615372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 03:15:07.615372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 27 03:15:07.615372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:07.615372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 27 03:15:07.615372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:07.615372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:07.615372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 27 03:15:07.615372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 27 03:15:07.615372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 27 03:15:07.615372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return ref.to_dict() May 27 03:15:07.615372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 03:15:07.615372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application next(self.gen) May 27 03:15:07.616762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 27 03:15:07.616762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application yield resource May 27 03:15:07.616762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 03:15:07.616762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application next(self.gen) May 27 03:15:07.616762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 27 03:15:07.616762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self.session.rollback() May 27 03:15:07.616762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 27 03:15:07.616762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self.force_reraise() May 27 03:15:07.616762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 27 03:15:07.616762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise self.value May 27 03:15:07.616762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 27 03:15:07.616762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 27 03:15:07.616762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 27 03:15:07.616762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application session.commit() May 27 03:15:07.616762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 27 03:15:07.616762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 27 03:15:07.616762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 27 03:15:07.617846 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._prepare_impl() May 27 03:15:07.617846 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 27 03:15:07.617846 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self.session.flush() May 27 03:15:07.617846 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 27 03:15:07.617846 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._flush(objects) May 27 03:15:07.617846 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 27 03:15:07.617846 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 27 03:15:07.617846 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 27 03:15:07.617846 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application compat.raise_( May 27 03:15:07.617846 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 03:15:07.617846 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception May 27 03:15:07.617846 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 27 03:15:07.617846 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application flush_context.execute() May 27 03:15:07.617846 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 27 03:15:07.617846 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rec.execute(self) May 27 03:15:07.617846 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 27 03:15:07.617846 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 27 03:15:07.618705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 27 03:15:07.618705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application _emit_insert_statements( May 27 03:15:07.618705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 27 03:15:07.618705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application c = connection._execute_20( May 27 03:15:07.618705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 27 03:15:07.618705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 27 03:15:07.618705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 27 03:15:07.618705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 27 03:15:07.618705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 27 03:15:07.618705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ret = self._execute_context( May 27 03:15:07.618705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 27 03:15:07.618705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 27 03:15:07.618705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 27 03:15:07.618705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 27 03:15:07.618705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 03:15:07.618705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception May 27 03:15:07.619563 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 03:15:07.619563 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 03:15:07.619563 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 03:15:07.619563 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 03:15:07.619563 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 03:15:07.619563 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application result = self._query(query) May 27 03:15:07.619563 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 03:15:07.619563 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application conn.query(q) May 27 03:15:07.619563 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 03:15:07.619563 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 03:15:07.619563 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 03:15:07.619563 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application result.read() May 27 03:15:07.619563 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 03:15:07.619563 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 03:15:07.619563 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 03:15:07.619563 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 03:15:07.621312 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 03:15:07.621312 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 03:15:07.621312 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 03:15:07.621312 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 03:15:07.621312 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 27 03:15:07.621312 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 27 03:15:07.621312 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [parameters: {'id': '15550dc6281a4359a56a9b9d3ae2e126', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 27 03:15:07.621312 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 27 03:15:07.621312 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:07.621312 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:15:07.621312 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:07.621312 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:07.621312 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:07.621312 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:07.621312 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:07.621312 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:07.623330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:07.623330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:07.623330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:07.623330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:07.623330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:07.623330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:07.623330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 27 03:15:07.623330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 27 03:15:07.623330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:07.623330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:07.623330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 27 03:15:07.623330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 27 03:15:07.623330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 27 03:15:07.623330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 27 03:15:07.623330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 27 03:15:07.623330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:07.625312 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 233/478] 10.222.0.24 () {60 vars in 1131 bytes} [Fri May 27 03:15:07 2022] POST /identity/v3/roles => generated 144 bytes in 236 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 27 03:15:07.627001 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ff460-7cce-46f7-a541-d10a564862aa None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:07.774864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ff460-7cce-46f7-a541-d10a564862aa None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:07.822407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4653e-dd79-4695-a266-213945e63f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:07.823784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4653e-dd79-4695-a266-213945e63f27 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:07.824339 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4653e-dd79-4695-a266-213945e63f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:07.824614 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4653e-dd79-4695-a266-213945e63f27 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:07.837038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4653e-dd79-4695-a266-213945e63f27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:07.845892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-aec4653e-dd79-4695-a266-213945e63f27 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:15:07.845892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:07.845892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:07.845892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:07.845892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:07.845892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:07.845892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:07.845892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:07.845892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:07.845892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:07.845892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:07.845892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:07.845892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:07.845892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:07.845892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:07.845892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:07.846784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:07.846784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:07.846784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:07.846784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:07.846784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:07.846784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:07.846784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:07.846784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:07.846784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:07.846784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:07.846784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:07.846784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:07.846784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:07.846784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:07.846784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:07.846784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:07.848244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:07.848244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:07.848244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:07.848244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:07.848244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:07.848244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:07.848244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:07.848244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:07.848244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:07.848244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:07.848244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:15:07.848244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:07.848244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 246/479] 10.222.0.24 () {56 vars in 1114 bytes} [Fri May 27 03:15:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:07.852733 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431a0b7-5717-4488-ab89-a2d85710d514 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:07.887914 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:07.981583 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431a0b7-5717-4488-ab89-a2d85710d514 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:08.029953 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab2116c-3a2f-4201-b928-b07dfececab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:08.030617 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2116c-3a2f-4201-b928-b07dfececab3 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:08.030939 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2116c-3a2f-4201-b928-b07dfececab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:08.031218 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab2116c-3a2f-4201-b928-b07dfececab3 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:08.031978 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab2116c-3a2f-4201-b928-b07dfececab3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:08.032281 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab2116c-3a2f-4201-b928-b07dfececab3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:08.033656 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab2116c-3a2f-4201-b928-b07dfececab3 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:08.045939 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 234/480] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:08.969059 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab4d27-d942-4eff-a0ac-f0d29e090375 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:08.969597 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab4d27-d942-4eff-a0ac-f0d29e090375 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:08.969597 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ab4d27-d942-4eff-a0ac-f0d29e090375 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:08.971571 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 247/481] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:08.976948 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a1664-19e6-44f4-909b-47bd52f3bc79 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:08.977361 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a1664-19e6-44f4-909b-47bd52f3bc79 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:08.977546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64a1664-19e6-44f4-909b-47bd52f3bc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:08.998133 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-c64a1664-19e6-44f4-909b-47bd52f3bc79 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:08.999338 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c64a1664-19e6-44f4-909b-47bd52f3bc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:09.100234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 235/482] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:15:09.136249 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab4d27-d942-4eff-a0ac-f0d29e090375 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:09.208879 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:09.286343 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ab4d27-d942-4eff-a0ac-f0d29e090375 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:09.334071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9399f7e1-e675-4e36-896c-b88ea4d95472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:09.335157 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399f7e1-e675-4e36-896c-b88ea4d95472 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:09.335516 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399f7e1-e675-4e36-896c-b88ea4d95472 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:09.335917 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9399f7e1-e675-4e36-896c-b88ea4d95472 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:09.355773 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9399f7e1-e675-4e36-896c-b88ea4d95472 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:09.363845 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-9399f7e1-e675-4e36-896c-b88ea4d95472 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:09.363845 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:09.363845 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:09.363845 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:09.363845 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:09.363845 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:09.363845 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:09.363845 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:09.363845 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:09.363845 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:09.363845 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:09.363845 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:09.363845 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:09.363845 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:09.363845 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:09.363845 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:09.364710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:09.364710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:09.364710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:09.364710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:09.364710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:09.364710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:09.364710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:09.364710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:09.364710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:09.364710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:09.364710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:09.364710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:09.364710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:09.364710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:09.364710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:09.364710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:09.365491 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:09.365491 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:09.365491 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:09.365491 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:09.365491 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:09.365491 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:09.365491 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:09.365491 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:09.365491 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:09.365491 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:09.365491 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:09.365491 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:09.367535 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 248/483] 10.222.0.24 () {56 vars in 1102 bytes} [Fri May 27 03:15:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:09.373981 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a1664-19e6-44f4-909b-47bd52f3bc79 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:09.392904 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:09.533781 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64a1664-19e6-44f4-909b-47bd52f3bc79 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:09.580966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ce589-1240-4884-8c01-97fb7bce03b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:09.581557 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ce589-1240-4884-8c01-97fb7bce03b1 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:09.581835 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ce589-1240-4884-8c01-97fb7bce03b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:09.582110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ce589-1240-4884-8c01-97fb7bce03b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:09.582864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ce589-1240-4884-8c01-97fb7bce03b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:09.583178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ce589-1240-4884-8c01-97fb7bce03b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:09.584563 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ce589-1240-4884-8c01-97fb7bce03b1 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:09.598175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 236/484] 10.222.0.24 () {56 vars in 1103 bytes} [Fri May 27 03:15:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:09.603322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9399f7e1-e675-4e36-896c-b88ea4d95472 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:09.770664 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:09.775587 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9399f7e1-e675-4e36-896c-b88ea4d95472 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:09.819739 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34521f58-97fc-42f0-a789-032dc614cd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:09.820545 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34521f58-97fc-42f0-a789-032dc614cd5f None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:09.820828 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34521f58-97fc-42f0-a789-032dc614cd5f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:09.821100 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34521f58-97fc-42f0-a789-032dc614cd5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:09.836283 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34521f58-97fc-42f0-a789-032dc614cd5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:09.837728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34521f58-97fc-42f0-a789-032dc614cd5f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:09.862980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-34521f58-97fc-42f0-a789-032dc614cd5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:09.862980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:09.862980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:09.862980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:09.862980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:09.862980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:09.862980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:09.862980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:09.862980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:09.862980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:09.862980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:09.862980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:09.862980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:09.862980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:09.862980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:09.863880 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:09.863880 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:09.863880 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:09.863880 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:09.863880 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:09.863880 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:09.863880 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:09.863880 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:09.863880 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:09.863880 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:09.863880 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:09.863880 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:09.863880 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:09.863880 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:09.863880 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:09.863880 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:09.863880 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:09.864721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:09.864721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:09.864721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:09.864721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:09.867017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 249/485] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:09.872760 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ce589-1240-4884-8c01-97fb7bce03b1 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:09.982030 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:09.993037 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd79ffb-d840-473e-82ae-b8188dcb06e4 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:09.993941 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd79ffb-d840-473e-82ae-b8188dcb06e4 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:09.994227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd79ffb-d840-473e-82ae-b8188dcb06e4 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:09.995645 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 250/486] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:15:10.001064 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db471b8-9277-430f-b182-1e3329267b06 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:10.001348 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db471b8-9277-430f-b182-1e3329267b06 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:10.001632 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db471b8-9277-430f-b182-1e3329267b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:10.022866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-9db471b8-9277-430f-b182-1e3329267b06 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:10.029441 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ce589-1240-4884-8c01-97fb7bce03b1 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:10.031329 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-9db471b8-9277-430f-b182-1e3329267b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:10.089887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bdbc2b-025f-4ec9-866a-e8559a84e72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:10.090497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdbc2b-025f-4ec9-866a-e8559a84e72d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:10.090779 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdbc2b-025f-4ec9-866a-e8559a84e72d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:10.091155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bdbc2b-025f-4ec9-866a-e8559a84e72d None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:10.092080 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bdbc2b-025f-4ec9-866a-e8559a84e72d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:10.092394 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bdbc2b-025f-4ec9-866a-e8559a84e72d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:10.093952 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bdbc2b-025f-4ec9-866a-e8559a84e72d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:10.108806 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 237/487] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:10.114205 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bdbc2b-025f-4ec9-866a-e8559a84e72d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:10.186371 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 251/488] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:15:10.226610 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db471b8-9277-430f-b182-1e3329267b06 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:10.271495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bdbc2b-025f-4ec9-866a-e8559a84e72d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:10.310888 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:10.317680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163406f-6a55-44af-b729-1fd298fee544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:10.321766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163406f-6a55-44af-b729-1fd298fee544 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:10.321766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163406f-6a55-44af-b729-1fd298fee544 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:10.321766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163406f-6a55-44af-b729-1fd298fee544 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:10.328631 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163406f-6a55-44af-b729-1fd298fee544 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:10.330486 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163406f-6a55-44af-b729-1fd298fee544 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:10.348599 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:10.349993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-4163406f-6a55-44af-b729-1fd298fee544 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:15:10.349993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:10.349993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:10.349993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:10.349993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:10.349993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:10.349993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:10.349993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:10.349993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:10.349993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:10.349993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:10.349993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:10.349993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:15:10.349993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:15:10.349993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:15:10.349993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:15:10.350826 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:10.350826 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:10.350826 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:15:10.350826 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:10.350826 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:15:10.350826 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:10.350826 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:10.350826 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:10.350826 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:10.350826 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:10.350826 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:10.350826 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:10.350826 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:10.350826 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:10.350826 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:10.350826 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:10.350826 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:10.350826 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:10.351995 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:10.351995 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:10.351995 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:15:10.351995 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:15:10.351995 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:15:10.351995 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:15:10.351995 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:15:10.351995 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:15:10.351995 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:15:10.351995 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:10.354602 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 238/489] 10.222.0.24 () {56 vars in 1093 bytes} [Fri May 27 03:15:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:15:10.356259 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b52f4c-0e28-4d54-898b-3f5821ffab25 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:10.356746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b52f4c-0e28-4d54-898b-3f5821ffab25 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:10.357210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b52f4c-0e28-4d54-898b-3f5821ffab25 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:10.362959 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 239/490] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:10.364073 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b52f4c-0e28-4d54-898b-3f5821ffab25 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:10.374642 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db471b8-9277-430f-b182-1e3329267b06 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:10.419709 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b9ee0-8eb2-4205-8408-fce8d659a2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:10.420649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b9ee0-8eb2-4205-8408-fce8d659a2b8 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:10.421198 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b9ee0-8eb2-4205-8408-fce8d659a2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:10.421477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b9ee0-8eb2-4205-8408-fce8d659a2b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:10.435103 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b9ee0-8eb2-4205-8408-fce8d659a2b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:10.445132 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-a89b9ee0-8eb2-4205-8408-fce8d659a2b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:10.445132 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:10.445132 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:10.445132 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:10.445132 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:10.445132 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:10.445132 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:10.445132 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:10.445132 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:10.445132 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:10.445132 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:10.445132 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:10.445132 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:10.445132 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:10.445132 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:10.445132 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:10.446052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:10.446052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:10.446052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:10.446052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:10.446052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:10.446052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:10.446052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:10.446052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:10.446052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:10.446052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:10.446052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:10.446052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:10.446052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:10.446052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:10.446052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:10.446052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:10.447241 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:10.447241 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:10.447241 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:10.447241 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:10.447241 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:10.447241 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:10.447241 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:10.447241 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:10.447241 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:10.447241 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:10.447241 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:10.447241 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:10.447241 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 252/491] 10.222.0.24 () {56 vars in 1102 bytes} [Fri May 27 03:15:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:10.451711 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fa588-c938-4c86-acb5-9f057f8db480 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:10.451711 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fa588-c938-4c86-acb5-9f057f8db480 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:10.451711 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283fa588-c938-4c86-acb5-9f057f8db480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:10.467286 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-283fa588-c938-4c86-acb5-9f057f8db480 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:10.468468 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-283fa588-c938-4c86-acb5-9f057f8db480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:10.471886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b52f4c-0e28-4d54-898b-3f5821ffab25 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:10.509537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03188609-e5f1-473b-b4ce-c323b7ee48f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:10.510206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03188609-e5f1-473b-b4ce-c323b7ee48f3 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:10.511076 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03188609-e5f1-473b-b4ce-c323b7ee48f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:10.511346 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03188609-e5f1-473b-b4ce-c323b7ee48f3 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:10.512184 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03188609-e5f1-473b-b4ce-c323b7ee48f3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:10.512488 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03188609-e5f1-473b-b4ce-c323b7ee48f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:10.513932 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03188609-e5f1-473b-b4ce-c323b7ee48f3 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:10.550922 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 253/492] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:15:10.550922 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efaa2e3-2f76-4103-94b5-cf47b225f6bd None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:10.550922 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efaa2e3-2f76-4103-94b5-cf47b225f6bd None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:10.550922 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efaa2e3-2f76-4103-94b5-cf47b225f6bd None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:10.550922 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 254/493] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:10.550922 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efaa2e3-2f76-4103-94b5-cf47b225f6bd None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:10.552204 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 240/494] 10.222.0.24 () {56 vars in 1133 bytes} [Fri May 27 03:15:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:10.558342 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180693dc-3c85-4629-8944-b50793645887 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:10.558644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180693dc-3c85-4629-8944-b50793645887 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:10.559014 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180693dc-3c85-4629-8944-b50793645887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:10.576251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-180693dc-3c85-4629-8944-b50793645887 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:10.577444 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-180693dc-3c85-4629-8944-b50793645887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:10.622733 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:10.631572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:10.646255 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 241/495] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:15:10.647017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180693dc-3c85-4629-8944-b50793645887 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:10.663528 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efaa2e3-2f76-4103-94b5-cf47b225f6bd None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:10.700650 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c980a-3c53-4fcd-bbf7-2c457641f81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:10.702245 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c980a-3c53-4fcd-bbf7-2c457641f81a None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:10.702440 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c980a-3c53-4fcd-bbf7-2c457641f81a None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:10.703124 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c980a-3c53-4fcd-bbf7-2c457641f81a None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:10.703868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c980a-3c53-4fcd-bbf7-2c457641f81a None admin] RBAC: Adding query filter params (name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:10.704173 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c980a-3c53-4fcd-bbf7-2c457641f81a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:10.705560 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c980a-3c53-4fcd-bbf7-2c457641f81a None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:10.718322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 255/496] 10.222.0.24 () {56 vars in 1103 bytes} [Fri May 27 03:15:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:10.718322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c980a-3c53-4fcd-bbf7-2c457641f81a None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:10.769543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180693dc-3c85-4629-8944-b50793645887 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:10.806412 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7c624-d420-463f-a407-7bcbb9b01ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:10.807101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7c624-d420-463f-a407-7bcbb9b01ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:10.807466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7c624-d420-463f-a407-7bcbb9b01ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:10.807749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7c624-d420-463f-a407-7bcbb9b01ca1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:10.824761 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c980a-3c53-4fcd-bbf7-2c457641f81a None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:10.852877 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:10.863036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0609d3b9-f1cf-48be-a12b-47eff2bffb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:10.863826 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609d3b9-f1cf-48be-a12b-47eff2bffb05 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:10.864184 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609d3b9-f1cf-48be-a12b-47eff2bffb05 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:10.864373 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609d3b9-f1cf-48be-a12b-47eff2bffb05 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:10.866679 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:10.876145 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a7c624-d420-463f-a407-7bcbb9b01ca1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:10.877196 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0609d3b9-f1cf-48be-a12b-47eff2bffb05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:10.877609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a7c624-d420-463f-a407-7bcbb9b01ca1 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:10.885865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-0609d3b9-f1cf-48be-a12b-47eff2bffb05 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:10.885865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:10.885865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:10.885865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:10.885865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:10.885865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:10.885865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:10.885865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:10.885865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:10.885865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:10.885865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:10.885865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:10.885865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:10.885865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:10.885865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:10.885865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:10.886774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:10.886774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:10.886774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:10.886774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:10.886774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:10.886774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:10.886774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:10.886774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:10.886774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:10.886774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:10.886774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:10.886774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:10.886774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:10.886774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:10.886774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:10.886774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:10.887672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:10.887672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:10.887672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:10.887672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:10.887672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:10.887672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:10.887672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:10.887672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:10.887672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:10.887672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:10.887672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:10.887672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:10.888271 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 256/497] 10.222.0.24 () {56 vars in 1102 bytes} [Fri May 27 03:15:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:10.888838 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0609d3b9-f1cf-48be-a12b-47eff2bffb05 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:10.983457 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 242/498] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:15:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:15:10.984295 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7c624-d420-463f-a407-7bcbb9b01ca1 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:11.009575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0609d3b9-f1cf-48be-a12b-47eff2bffb05 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:11.042708 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8d3be-5f9e-4b78-b1a2-6f885af1738f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:11.043516 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8d3be-5f9e-4b78-b1a2-6f885af1738f None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:11.043795 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8d3be-5f9e-4b78-b1a2-6f885af1738f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:11.044065 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8d3be-5f9e-4b78-b1a2-6f885af1738f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:11.054096 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b8d3be-5f9e-4b78-b1a2-6f885af1738f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:11.059556 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:11.062322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-c9b8d3be-5f9e-4b78-b1a2-6f885af1738f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:11.062322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:11.062322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:11.062322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:11.062322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:11.062322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:11.062322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:11.062322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:11.062322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:11.062322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:11.062322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:11.062322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:11.062322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:11.062322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:11.062322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:11.062322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:11.063441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:11.063441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:11.063441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:11.063441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:11.063441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:11.063441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:11.063441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:11.063441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:11.063441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:11.063441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:11.063441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:11.063441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:11.063441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:11.063441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:11.063441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:11.063441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:11.064217 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:11.064217 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:11.064217 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:11.064217 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:11.064217 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:11.064217 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:11.064217 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:11.064217 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:11.064217 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:11.064217 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:11.064217 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:11.064217 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:11.064217 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 257/499] 10.222.0.24 () {56 vars in 1102 bytes} [Fri May 27 03:15:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:11.064858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8d3be-5f9e-4b78-b1a2-6f885af1738f None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:11.087675 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7c624-d420-463f-a407-7bcbb9b01ca1 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:11.118674 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:11.121692 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3064f-8a04-4b26-b4c8-1ececc221c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:11.122480 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3064f-8a04-4b26-b4c8-1ececc221c5e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:11.122773 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3064f-8a04-4b26-b4c8-1ececc221c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:11.123071 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3064f-8a04-4b26-b4c8-1ececc221c5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:11.133191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3064f-8a04-4b26-b4c8-1ececc221c5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:11.134588 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3064f-8a04-4b26-b4c8-1ececc221c5e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:11.152943 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-2ed3064f-8a04-4b26-b4c8-1ececc221c5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:11.152943 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:11.152943 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:11.152943 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:11.152943 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:11.152943 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:11.152943 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:11.152943 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:11.152943 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:11.152943 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:11.152943 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:11.152943 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:11.152943 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:11.152943 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:11.152943 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:11.153887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:11.153887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:11.153887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:11.153887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:11.153887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:11.153887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:11.153887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:11.153887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:11.153887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:11.153887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:11.153887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:11.153887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:11.153887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:11.153887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:11.153887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:11.153887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:11.153887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:11.154728 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:11.154728 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:11.154728 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:11.154728 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:11.154728 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 243/500] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:11.155710 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4525097-4ce4-4c3f-b86d-7663458d5362 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:11.155985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4525097-4ce4-4c3f-b86d-7663458d5362 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:11.156254 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4525097-4ce4-4c3f-b86d-7663458d5362 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:11.157656 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 244/501] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:11.158165 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4525097-4ce4-4c3f-b86d-7663458d5362 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:11.167776 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8d3be-5f9e-4b78-b1a2-6f885af1738f None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:11.201806 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76336a57-40b5-459a-8f41-ea2dd856df00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:11.202466 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76336a57-40b5-459a-8f41-ea2dd856df00 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:11.202750 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76336a57-40b5-459a-8f41-ea2dd856df00 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:11.203049 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76336a57-40b5-459a-8f41-ea2dd856df00 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:11.203781 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76336a57-40b5-459a-8f41-ea2dd856df00 None admin] RBAC: Adding query filter params (name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:11.204085 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76336a57-40b5-459a-8f41-ea2dd856df00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:11.205547 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76336a57-40b5-459a-8f41-ea2dd856df00 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:11.214066 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 258/502] 10.222.0.24 () {56 vars in 1103 bytes} [Fri May 27 03:15:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:11.214854 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76336a57-40b5-459a-8f41-ea2dd856df00 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:11.260639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4525097-4ce4-4c3f-b86d-7663458d5362 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:11.292754 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:11.298299 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f5922-c7ec-4679-a3f8-101b420ea26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:11.299125 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f5922-c7ec-4679-a3f8-101b420ea26b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:11.299409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f5922-c7ec-4679-a3f8-101b420ea26b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:11.299682 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f5922-c7ec-4679-a3f8-101b420ea26b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:11.309647 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f5922-c7ec-4679-a3f8-101b420ea26b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:11.311136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f5922-c7ec-4679-a3f8-101b420ea26b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:11.313698 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76336a57-40b5-459a-8f41-ea2dd856df00 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:11.328324 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-ac1f5922-c7ec-4679-a3f8-101b420ea26b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:11.328324 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:11.328324 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:11.328324 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:11.328324 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:11.328324 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:11.328324 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:11.328324 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:11.328324 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:11.328324 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:11.328324 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:11.328324 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:11.328324 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:11.328324 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:11.328324 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:11.329296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:11.329296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:11.329296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:11.329296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:11.329296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:11.329296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:11.329296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:11.329296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:11.329296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:11.329296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:11.329296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:11.329296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:11.329296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:11.329296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:11.329296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:11.329296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:11.329296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:11.330163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:11.330163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:11.330163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:11.330163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:11.330383 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:11.330477 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 245/503] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:11.330884 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f5922-c7ec-4679-a3f8-101b420ea26b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:11.347686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbc146-3073-41ba-9195-c40e40f70a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:11.348409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbc146-3073-41ba-9195-c40e40f70a37 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:11.348691 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbc146-3073-41ba-9195-c40e40f70a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:11.348971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbc146-3073-41ba-9195-c40e40f70a37 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:11.349709 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbc146-3073-41ba-9195-c40e40f70a37 None admin] RBAC: Adding query filter params (name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:11.350017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbc146-3073-41ba-9195-c40e40f70a37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:11.351427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbc146-3073-41ba-9195-c40e40f70a37 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:11.364611 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 259/504] 10.222.0.24 () {56 vars in 1103 bytes} [Fri May 27 03:15:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:11.364611 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08497efc-864a-43c0-8d01-29398bbc8995 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:11.364611 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08497efc-864a-43c0-8d01-29398bbc8995 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:11.364611 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08497efc-864a-43c0-8d01-29398bbc8995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:11.378214 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-08497efc-864a-43c0-8d01-29398bbc8995 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:11.379425 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-08497efc-864a-43c0-8d01-29398bbc8995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:11.436227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f5922-c7ec-4679-a3f8-101b420ea26b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:11.449855 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 260/505] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:15:11.450954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08497efc-864a-43c0-8d01-29398bbc8995 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:11.472250 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611d25f-086a-4553-a9d3-5363fc159157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:11.472916 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611d25f-086a-4553-a9d3-5363fc159157 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:11.473203 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611d25f-086a-4553-a9d3-5363fc159157 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:11.473812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611d25f-086a-4553-a9d3-5363fc159157 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:11.474549 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e611d25f-086a-4553-a9d3-5363fc159157 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:11.474901 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e611d25f-086a-4553-a9d3-5363fc159157 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:11.476281 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e611d25f-086a-4553-a9d3-5363fc159157 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:11.487876 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 246/506] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:11.488678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611d25f-086a-4553-a9d3-5363fc159157 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:11.529842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:11.554640 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08497efc-864a-43c0-8d01-29398bbc8995 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:11.563296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:11.585377 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611d25f-086a-4553-a9d3-5363fc159157 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:11.587375 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292fe6a-ad4c-43ce-bf95-ff115b00bde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:11.588161 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292fe6a-ad4c-43ce-bf95-ff115b00bde8 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:11.588490 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292fe6a-ad4c-43ce-bf95-ff115b00bde8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:11.588761 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0292fe6a-ad4c-43ce-bf95-ff115b00bde8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:11.599318 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0292fe6a-ad4c-43ce-bf95-ff115b00bde8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:11.600756 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0292fe6a-ad4c-43ce-bf95-ff115b00bde8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:11.618130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-0292fe6a-ad4c-43ce-bf95-ff115b00bde8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:11.618130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:11.618130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:11.618130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:11.618130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:11.618130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:11.618130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:11.618130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:11.618130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:11.618130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:11.618130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:11.618130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:11.618130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:11.618130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:11.618130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:11.619014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:11.619014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:11.619014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:11.619014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:11.619014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:11.619014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:11.619014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:11.619014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:11.619014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:11.619014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:11.619014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:11.619014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:11.619014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:11.619014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:11.619014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:11.619014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:11.619014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:11.620439 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:11.620439 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:11.620439 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:11.620439 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:11.620660 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a973358-deb6-4bb6-bf58-268a896ca09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:11.620660 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a973358-deb6-4bb6-bf58-268a896ca09c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:11.620660 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a973358-deb6-4bb6-bf58-268a896ca09c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:11.620871 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 261/507] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:11.620936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a973358-deb6-4bb6-bf58-268a896ca09c None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:11.620936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a973358-deb6-4bb6-bf58-268a896ca09c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:11.621102 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292fe6a-ad4c-43ce-bf95-ff115b00bde8 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:11.621179 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a973358-deb6-4bb6-bf58-268a896ca09c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:11.622454 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a973358-deb6-4bb6-bf58-268a896ca09c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:11.639172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 247/508] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:11.639172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a973358-deb6-4bb6-bf58-268a896ca09c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:11.729907 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0292fe6a-ad4c-43ce-bf95-ff115b00bde8 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:11.748342 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a973358-deb6-4bb6-bf58-268a896ca09c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:11.749864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:11.780153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb323ab8-014f-452b-b8bb-c720fb825f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:11.780941 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb323ab8-014f-452b-b8bb-c720fb825f05 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:11.781217 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb323ab8-014f-452b-b8bb-c720fb825f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:11.781485 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb323ab8-014f-452b-b8bb-c720fb825f05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:11.781638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42138f9-f3de-4301-9f0e-4fa2a262cf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:11.782418 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42138f9-f3de-4301-9f0e-4fa2a262cf3b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:11.782695 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42138f9-f3de-4301-9f0e-4fa2a262cf3b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:11.782996 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42138f9-f3de-4301-9f0e-4fa2a262cf3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:11.792353 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb323ab8-014f-452b-b8bb-c720fb825f05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:11.793217 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42138f9-f3de-4301-9f0e-4fa2a262cf3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:11.793791 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb323ab8-014f-452b-b8bb-c720fb825f05 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:11.794651 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42138f9-f3de-4301-9f0e-4fa2a262cf3b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:11.810810 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-eb323ab8-014f-452b-b8bb-c720fb825f05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:11.810810 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:11.810810 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:11.810810 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:11.810810 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:11.810810 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:11.810810 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:11.810810 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:11.810810 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:11.810810 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:11.810810 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:11.810810 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:11.810810 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:11.810810 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:11.810810 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:11.811705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:11.811705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:11.811705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:11.811705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:11.811705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:11.811705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:11.811705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:11.811705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:11.811705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:11.811705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:11.811705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:11.811705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:11.811705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:11.811705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:11.811705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:11.811705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:11.811705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:11.812542 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:11.812542 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:11.812542 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:11.812542 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:11.812749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-e42138f9-f3de-4301-9f0e-4fa2a262cf3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:11.812749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:11.812749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:11.812749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:11.812749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:11.812749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:11.812749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:11.812749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:11.812749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:11.812749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:11.812749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:11.812749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:11.812749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:11.812749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:11.812749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:11.813491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:11.813491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:11.813491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:11.813491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:11.813491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:11.813491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:11.813491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:11.813491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:11.813491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:11.813491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:11.813491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:11.813491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:11.813491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:11.813491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:11.813491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:11.813491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:11.813491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:11.814315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:11.814315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:11.814315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:11.814315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:11.814527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 262/509] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:11.814527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb323ab8-014f-452b-b8bb-c720fb825f05 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:11.817326 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 248/510] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:11.818051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42138f9-f3de-4301-9f0e-4fa2a262cf3b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:11.830728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:11.914608 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42138f9-f3de-4301-9f0e-4fa2a262cf3b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:11.915093 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb323ab8-014f-452b-b8bb-c720fb825f05 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:11.948790 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ab51d-3eda-4ba6-82da-3173a14fd72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:11.949272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85610c-e816-4976-b52e-7da2d6abb09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:11.949453 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ab51d-3eda-4ba6-82da-3173a14fd72f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:11.949735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ab51d-3eda-4ba6-82da-3173a14fd72f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:11.950008 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ab51d-3eda-4ba6-82da-3173a14fd72f None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:11.950097 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85610c-e816-4976-b52e-7da2d6abb09f None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:11.950313 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85610c-e816-4976-b52e-7da2d6abb09f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:11.950581 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85610c-e816-4976-b52e-7da2d6abb09f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:11.951006 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ab51d-3eda-4ba6-82da-3173a14fd72f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:11.951317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ab51d-3eda-4ba6-82da-3173a14fd72f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:11.952707 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ab51d-3eda-4ba6-82da-3173a14fd72f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:11.958481 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a85610c-e816-4976-b52e-7da2d6abb09f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:11.960125 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a85610c-e816-4976-b52e-7da2d6abb09f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:11.964158 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 249/511] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:11.965062 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ab51d-3eda-4ba6-82da-3173a14fd72f None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:11.976303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-0a85610c-e816-4976-b52e-7da2d6abb09f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 27 03:15:11.976303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:11.976303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:11.976303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:11.976303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:11.976303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:11.976303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:11.976303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:11.976303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:11.976303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:11.976303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:11.976303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:11.976303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:15:11.976303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:15:11.976303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:15:11.976303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:15:11.983315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:11.983315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:11.983315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:15:11.983315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:11.983315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:15:11.983315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:11.983315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:11.983315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:11.983315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:11.983315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:11.983315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:11.983315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:11.983315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:11.983315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:11.983315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:11.983315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:11.983315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:11.983315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:11.984215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:11.984215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:11.984215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:15:11.984215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:15:11.984215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:15:11.984215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:15:11.984215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:15:11.984215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:15:11.984215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 27 03:15:11.984215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:11.984215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 263/512] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:15:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:11.984215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85610c-e816-4976-b52e-7da2d6abb09f None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:11.985991 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:12.049349 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:12.078167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ab51d-3eda-4ba6-82da-3173a14fd72f None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:12.095649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85610c-e816-4976-b52e-7da2d6abb09f None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:12.112280 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234367dc-90b0-4114-a1ef-b6d0eade0b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:12.113071 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234367dc-90b0-4114-a1ef-b6d0eade0b45 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:12.113357 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234367dc-90b0-4114-a1ef-b6d0eade0b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:12.113635 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234367dc-90b0-4114-a1ef-b6d0eade0b45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:12.120948 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234367dc-90b0-4114-a1ef-b6d0eade0b45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:12.122380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234367dc-90b0-4114-a1ef-b6d0eade0b45 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:12.129792 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6578111c-bda4-4c46-a123-30f5d0b2ad91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:12.130442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578111c-bda4-4c46-a123-30f5d0b2ad91 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:12.130723 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578111c-bda4-4c46-a123-30f5d0b2ad91 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:12.131038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578111c-bda4-4c46-a123-30f5d0b2ad91 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:12.131751 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6578111c-bda4-4c46-a123-30f5d0b2ad91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:12.132055 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6578111c-bda4-4c46-a123-30f5d0b2ad91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:12.133406 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6578111c-bda4-4c46-a123-30f5d0b2ad91 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:12.136976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-234367dc-90b0-4114-a1ef-b6d0eade0b45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:12.136976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:12.136976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:12.136976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:12.136976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:12.136976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:12.136976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:12.136976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:12.136976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:12.136976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:12.136976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:12.136976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:12.136976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:15:12.136976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:15:12.136976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:15:12.137779 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:15:12.137779 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:12.137779 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:12.137779 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:12.137779 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:12.137779 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:12.137779 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:12.137779 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:12.137779 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:12.137779 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:12.137779 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:12.137779 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:12.137779 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:12.137779 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:12.137779 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:12.137779 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:12.137779 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:12.138623 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:15:12.138623 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:15:12.138623 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:15:12.138623 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:15:12.138623 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:15:12.138623 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:15:12.138623 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:12.138623 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:12.138623 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 250/513] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:12.139678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234367dc-90b0-4114-a1ef-b6d0eade0b45 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:12.148090 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 264/514] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:12.148904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6578111c-bda4-4c46-a123-30f5d0b2ad91 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:12.215868 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:12.239997 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234367dc-90b0-4114-a1ef-b6d0eade0b45 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:12.246557 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6578111c-bda4-4c46-a123-30f5d0b2ad91 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:12.273552 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903529bd-7b50-457a-a735-22cb1ce22230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:12.274209 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903529bd-7b50-457a-a735-22cb1ce22230 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:12.274490 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903529bd-7b50-457a-a735-22cb1ce22230 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:12.274764 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903529bd-7b50-457a-a735-22cb1ce22230 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:12.275511 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903529bd-7b50-457a-a735-22cb1ce22230 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:12.275817 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903529bd-7b50-457a-a735-22cb1ce22230 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:12.276007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:12.277245 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903529bd-7b50-457a-a735-22cb1ce22230 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:12.280788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fa441-c75a-4785-8934-ef7b409116b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:12.281567 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fa441-c75a-4785-8934-ef7b409116b0 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:12.283965 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fa441-c75a-4785-8934-ef7b409116b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:12.283965 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739fa441-c75a-4785-8934-ef7b409116b0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:12.288971 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 251/515] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:12.289764 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903529bd-7b50-457a-a735-22cb1ce22230 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:12.289940 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739fa441-c75a-4785-8934-ef7b409116b0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:12.291414 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739fa441-c75a-4785-8934-ef7b409116b0 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:12.308757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-739fa441-c75a-4785-8934-ef7b409116b0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:15:12.308757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:12.308757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:12.308757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:12.308757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:12.308757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:12.308757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:12.308757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:12.308757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:12.308757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:12.308757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:12.308757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:12.308757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:15:12.308757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:15:12.308757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:15:12.308757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:15:12.309843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:12.309843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:12.309843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:15:12.309843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:12.309843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:15:12.309843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:12.309843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:12.309843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:12.309843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:12.309843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:12.309843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:12.309843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:12.309843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:12.309843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:12.309843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:12.309843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:12.309843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:12.309843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:12.310956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:12.310956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:12.310956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:15:12.310956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:15:12.310956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:15:12.310956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:15:12.310956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:15:12.310956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:15:12.310956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:15:12.310956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:12.313094 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 265/516] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:15:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:12.314142 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fa441-c75a-4785-8934-ef7b409116b0 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:12.394536 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903529bd-7b50-457a-a735-22cb1ce22230 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:12.411369 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739fa441-c75a-4785-8934-ef7b409116b0 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:12.427268 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a47668-cbef-4802-bf88-559b173f1ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:12.427934 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a47668-cbef-4802-bf88-559b173f1ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:12.428218 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a47668-cbef-4802-bf88-559b173f1ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:12.428494 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a47668-cbef-4802-bf88-559b173f1ed8 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:12.429322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a47668-cbef-4802-bf88-559b173f1ed8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:12.429631 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a47668-cbef-4802-bf88-559b173f1ed8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:12.431086 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a47668-cbef-4802-bf88-559b173f1ed8 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:12.432699 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:12.444454 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7a9f7-f9e6-4245-b96b-6eaee4398299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:12.445109 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7a9f7-f9e6-4245-b96b-6eaee4398299 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:12.445390 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7a9f7-f9e6-4245-b96b-6eaee4398299 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:12.445662 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7a9f7-f9e6-4245-b96b-6eaee4398299 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:12.446439 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d7a9f7-f9e6-4245-b96b-6eaee4398299 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:12.446747 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d7a9f7-f9e6-4245-b96b-6eaee4398299 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:12.448417 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 252/517] 10.222.0.24 () {56 vars in 1137 bytes} [Fri May 27 03:15:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:12.448748 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d7a9f7-f9e6-4245-b96b-6eaee4398299 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:12.449213 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a47668-cbef-4802-bf88-559b173f1ed8 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:12.467545 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 266/518] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:15:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:12.468354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7a9f7-f9e6-4245-b96b-6eaee4398299 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:12.521007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:12.547351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a47668-cbef-4802-bf88-559b173f1ed8 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:12.566782 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7a9f7-f9e6-4245-b96b-6eaee4398299 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:12.581558 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca399a40-9af3-4a5c-9648-8e5fb916a645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:12.582345 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca399a40-9af3-4a5c-9648-8e5fb916a645 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:12.582620 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca399a40-9af3-4a5c-9648-8e5fb916a645 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:12.582919 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca399a40-9af3-4a5c-9648-8e5fb916a645 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:12.590304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca399a40-9af3-4a5c-9648-8e5fb916a645 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:12.591785 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca399a40-9af3-4a5c-9648-8e5fb916a645 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:12.599285 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c83821-5de7-48d5-a261-8e5ba90b1bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:12.599947 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c83821-5de7-48d5-a261-8e5ba90b1bd7 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:12.600228 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c83821-5de7-48d5-a261-8e5ba90b1bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:12.600499 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c83821-5de7-48d5-a261-8e5ba90b1bd7 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:12.601191 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c83821-5de7-48d5-a261-8e5ba90b1bd7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:12.602598 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c83821-5de7-48d5-a261-8e5ba90b1bd7 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:12.606380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-ca399a40-9af3-4a5c-9648-8e5fb916a645 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 27 03:15:12.606380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:12.606380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:12.606380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:12.606380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:12.606380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:12.606380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:12.606380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:12.606380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:12.606380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:12.606380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:12.606380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:12.606380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:15:12.606380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:15:12.606380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:15:12.606380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:15:12.607475 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:12.607475 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:12.607475 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:15:12.607475 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:12.607475 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:15:12.607475 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:12.607475 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:12.607475 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:12.607475 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:12.607475 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:12.607475 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:12.607475 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:12.607475 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:12.607475 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:12.607475 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:12.607475 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:12.607475 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:12.607475 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:12.608505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:12.608505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:12.608505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:15:12.608505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:15:12.608505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:15:12.608505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:15:12.608505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:15:12.608505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:15:12.608505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 27 03:15:12.608505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:12.608505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 253/519] 10.222.0.24 () {56 vars in 1102 bytes} [Fri May 27 03:15:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:12.608505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca399a40-9af3-4a5c-9648-8e5fb916a645 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:12.657881 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:12.701956 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca399a40-9af3-4a5c-9648-8e5fb916a645 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:12.715452 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 267/520] 10.222.0.24 () {60 vars in 1132 bytes} [Fri May 27 03:15:12 2022] POST /identity/v3/users => generated 247 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:15:12.716381 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c83821-5de7-48d5-a261-8e5ba90b1bd7 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:12.740318 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d849b43-ba7d-4e9b-b966-92a9c51a14d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:12.741018 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d849b43-ba7d-4e9b-b966-92a9c51a14d7 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:12.742056 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d849b43-ba7d-4e9b-b966-92a9c51a14d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:12.742285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d849b43-ba7d-4e9b-b966-92a9c51a14d7 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:12.743506 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d849b43-ba7d-4e9b-b966-92a9c51a14d7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:12.743828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d849b43-ba7d-4e9b-b966-92a9c51a14d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:12.746166 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d849b43-ba7d-4e9b-b966-92a9c51a14d7 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:12.770108 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 254/521] 10.222.0.24 () {56 vars in 1137 bytes} [Fri May 27 03:15:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:12.771083 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d849b43-ba7d-4e9b-b966-92a9c51a14d7 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:12.837805 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c83821-5de7-48d5-a261-8e5ba90b1bd7 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:12.847903 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:12.876166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503227fe-9e93-4d5a-bd27-10e483d5a4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:12.876833 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503227fe-9e93-4d5a-bd27-10e483d5a4a7 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:12.877113 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503227fe-9e93-4d5a-bd27-10e483d5a4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:12.877387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503227fe-9e93-4d5a-bd27-10e483d5a4a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:12.887301 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d849b43-ba7d-4e9b-b966-92a9c51a14d7 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:12.898977 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:12.923467 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86444ae2-e46d-4483-b634-48ecbc11a155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:12.924119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86444ae2-e46d-4483-b634-48ecbc11a155 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:12.924397 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86444ae2-e46d-4483-b634-48ecbc11a155 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:12.924667 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86444ae2-e46d-4483-b634-48ecbc11a155 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:12.925491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86444ae2-e46d-4483-b634-48ecbc11a155 None admin] RBAC: Adding query filter params (role.id=8e315ffd561a48f0891ef53b51d548d8, scope.project.id=6ceffdd955324aec84ba048d2137ae00, user.id=22a9592143014f3287cd75c60ca17237) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:12.925798 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86444ae2-e46d-4483-b634-48ecbc11a155 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:12.927263 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86444ae2-e46d-4483-b634-48ecbc11a155 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:12.933214 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 255/522] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:15:12 2022] GET /identity/v3/role_assignments?user.id=22a9592143014f3287cd75c60ca17237&scope.project.id=6ceffdd955324aec84ba048d2137ae00&role.id=8e315ffd561a48f0891ef53b51d548d8 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:12.933840 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86444ae2-e46d-4483-b634-48ecbc11a155 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:12.958514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503227fe-9e93-4d5a-bd27-10e483d5a4a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:12.960330 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503227fe-9e93-4d5a-bd27-10e483d5a4a7 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:13.067106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86444ae2-e46d-4483-b634-48ecbc11a155 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:13.086876 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 268/523] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:15:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:15:13.086876 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503227fe-9e93-4d5a-bd27-10e483d5a4a7 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:13.095689 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:13.113038 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1844d677-212c-46f0-8f63-19fb821622f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:13.113725 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844d677-212c-46f0-8f63-19fb821622f5 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:13.114006 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844d677-212c-46f0-8f63-19fb821622f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:13.114284 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1844d677-212c-46f0-8f63-19fb821622f5 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:13.115174 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1844d677-212c-46f0-8f63-19fb821622f5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:13.115484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1844d677-212c-46f0-8f63-19fb821622f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:13.116958 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1844d677-212c-46f0-8f63-19fb821622f5 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:13.138619 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 256/524] 10.222.0.24 () {56 vars in 1139 bytes} [Fri May 27 03:15:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:13.139512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1844d677-212c-46f0-8f63-19fb821622f5 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:13.173536 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:13.195888 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503227fe-9e93-4d5a-bd27-10e483d5a4a7 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:13.231526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6112ff8-fc4d-4d48-a350-4e5670c0e16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:13.232180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6112ff8-fc4d-4d48-a350-4e5670c0e16c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:13.232453 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6112ff8-fc4d-4d48-a350-4e5670c0e16c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:13.232713 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6112ff8-fc4d-4d48-a350-4e5670c0e16c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:13.251444 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1844d677-212c-46f0-8f63-19fb821622f5 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:13.289262 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34621cae-8817-48ac-a8a9-8d64ee9cb60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:13.290441 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34621cae-8817-48ac-a8a9-8d64ee9cb60f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:13.290960 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34621cae-8817-48ac-a8a9-8d64ee9cb60f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:13.291378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34621cae-8817-48ac-a8a9-8d64ee9cb60f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:13.302815 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34621cae-8817-48ac-a8a9-8d64ee9cb60f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:13.303028 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6112ff8-fc4d-4d48-a350-4e5670c0e16c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:13.304610 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6112ff8-fc4d-4d48-a350-4e5670c0e16c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:13.304793 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34621cae-8817-48ac-a8a9-8d64ee9cb60f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:13.315103 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:13.326542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-34621cae-8817-48ac-a8a9-8d64ee9cb60f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:13.326542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:13.326542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:13.326542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:13.326542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:13.326542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:13.326542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:13.326542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:13.326542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:13.326542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:13.326542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:13.326542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:13.326542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:13.326542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:13.326542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:13.327702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:13.327702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:13.327702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:13.327702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:13.327702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:13.327702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:13.327702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:13.327702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:13.327702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:13.327702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:13.327702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:13.327702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:13.327702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:13.327702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:13.327702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:13.327702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:13.327702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:13.328703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:13.328703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:13.328703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:13.328703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:13.329887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 257/525] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:13.330820 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34621cae-8817-48ac-a8a9-8d64ee9cb60f None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:13.416678 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 269/526] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:15:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:15:13.417635 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6112ff8-fc4d-4d48-a350-4e5670c0e16c None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:13.456450 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:13.474405 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34621cae-8817-48ac-a8a9-8d64ee9cb60f None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:13.520644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7adb8e-3cca-4698-8aa2-9f72baf90cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:13.521581 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7adb8e-3cca-4698-8aa2-9f72baf90cae None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:13.521872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7adb8e-3cca-4698-8aa2-9f72baf90cae None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:13.522151 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7adb8e-3cca-4698-8aa2-9f72baf90cae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:13.523223 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6112ff8-fc4d-4d48-a350-4e5670c0e16c None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:13.534126 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:13.560024 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14730f-d98a-4507-9835-cf15407a29c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:13.560686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14730f-d98a-4507-9835-cf15407a29c9 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:13.561012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14730f-d98a-4507-9835-cf15407a29c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:13.561286 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed14730f-d98a-4507-9835-cf15407a29c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:13.562001 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14730f-d98a-4507-9835-cf15407a29c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:13.562305 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14730f-d98a-4507-9835-cf15407a29c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:13.563770 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed14730f-d98a-4507-9835-cf15407a29c9 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:13.577752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 270/527] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:13.578950 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14730f-d98a-4507-9835-cf15407a29c9 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:13.621063 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7adb8e-3cca-4698-8aa2-9f72baf90cae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:13.622891 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7adb8e-3cca-4698-8aa2-9f72baf90cae None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:13.715372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:13.726120 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed14730f-d98a-4507-9835-cf15407a29c9 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:13.737398 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 258/528] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:15:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:15:13.741343 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7adb8e-3cca-4698-8aa2-9f72baf90cae None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:13.770582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5080e-345d-4da3-9869-f60316cf7fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:13.771423 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5080e-345d-4da3-9869-f60316cf7fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:13.771973 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5080e-345d-4da3-9869-f60316cf7fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:13.772082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5080e-345d-4da3-9869-f60316cf7fc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:13.785340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df5080e-345d-4da3-9869-f60316cf7fc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:13.786937 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df5080e-345d-4da3-9869-f60316cf7fc8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:13.806346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-1df5080e-345d-4da3-9869-f60316cf7fc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:13.806346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:13.806346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:13.806346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:13.806346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:13.806346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:13.806346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:13.806346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:13.806346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:13.806346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:13.806346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:13.806346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:13.806346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:13.806346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:13.806346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:13.807387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:13.807387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:13.807387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:13.807387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:13.807387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:13.807387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:13.807387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:13.807387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:13.807387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:13.807387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:13.807387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:13.807387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:13.807387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:13.807387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:13.807387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:13.807387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:13.807387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:13.808217 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:13.808217 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:13.808217 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:13.808217 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:13.813011 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 271/529] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:13.813977 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5080e-345d-4da3-9869-f60316cf7fc8 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:13.841281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:13.865982 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7adb8e-3cca-4698-8aa2-9f72baf90cae None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:13.903155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d414c-bd00-4d24-acfa-4eaca5c76615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:13.903929 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d414c-bd00-4d24-acfa-4eaca5c76615 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:13.904371 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d414c-bd00-4d24-acfa-4eaca5c76615 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:13.904647 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d414c-bd00-4d24-acfa-4eaca5c76615 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:13.911580 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d414c-bd00-4d24-acfa-4eaca5c76615 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:13.913784 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d414c-bd00-4d24-acfa-4eaca5c76615 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:13.928499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-4d1d414c-bd00-4d24-acfa-4eaca5c76615 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:13.928499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:13.928499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:13.928499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:13.928499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:13.928499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:13.928499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:13.928499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:13.928499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:13.928499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:13.928499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:13.928499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:13.928499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:15:13.928499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:15:13.928499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:15:13.929378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:15:13.929378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:13.929378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:13.929378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:13.929378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:13.929378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:13.929378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:13.929378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:13.929378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:13.929378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:13.929378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:13.929378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:13.929378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:13.929378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:13.929378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:13.929378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:13.929378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:13.930198 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:15:13.930198 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:15:13.930198 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:15:13.930198 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:15:13.930198 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:15:13.930198 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:15:13.930198 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:13.930198 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:13.930198 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 259/530] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:13.931018 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d414c-bd00-4d24-acfa-4eaca5c76615 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:13.955614 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5080e-345d-4da3-9869-f60316cf7fc8 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:13.986650 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:14.009182 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e53206-c054-4c5e-939f-d796a4d4af79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:14.010413 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e53206-c054-4c5e-939f-d796a4d4af79 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:14.010864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e53206-c054-4c5e-939f-d796a4d4af79 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:14.011797 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e53206-c054-4c5e-939f-d796a4d4af79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:14.027485 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e53206-c054-4c5e-939f-d796a4d4af79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:14.029188 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e53206-c054-4c5e-939f-d796a4d4af79 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:14.051446 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d414c-bd00-4d24-acfa-4eaca5c76615 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:14.054788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-22e53206-c054-4c5e-939f-d796a4d4af79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:14.054788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:14.054788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:14.054788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:14.054788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:14.054788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:14.054788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:14.054788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:14.054788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:14.054788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:14.054788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:14.054788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:14.054788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:14.054788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:14.054788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:14.055836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:14.055836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:14.055836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:14.055836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:14.055836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:14.055836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:14.055836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:14.055836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:14.055836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:14.055836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:14.055836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:14.055836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:14.055836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:14.055836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:14.055836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:14.055836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:14.055836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:14.056662 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:14.056662 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:14.056662 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:14.056662 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:14.058248 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 272/531] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:14.059233 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e53206-c054-4c5e-939f-d796a4d4af79 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:14.111428 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75edd0bf-d4b7-47e0-97ae-563d0f6cb117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:14.111976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edd0bf-d4b7-47e0-97ae-563d0f6cb117 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:14.112233 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edd0bf-d4b7-47e0-97ae-563d0f6cb117 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:14.112515 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75edd0bf-d4b7-47e0-97ae-563d0f6cb117 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:14.113931 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75edd0bf-d4b7-47e0-97ae-563d0f6cb117 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:14.113931 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75edd0bf-d4b7-47e0-97ae-563d0f6cb117 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:14.115050 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75edd0bf-d4b7-47e0-97ae-563d0f6cb117 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:14.128857 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 260/532] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:14.130052 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75edd0bf-d4b7-47e0-97ae-563d0f6cb117 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:14.162714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:14.194539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e53206-c054-4c5e-939f-d796a4d4af79 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:14.232248 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc548ac-6d2c-48d0-abb4-9cfcba3fa3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:14.232914 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc548ac-6d2c-48d0-abb4-9cfcba3fa3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:14.233194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc548ac-6d2c-48d0-abb4-9cfcba3fa3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:14.233469 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc548ac-6d2c-48d0-abb4-9cfcba3fa3f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:14.234420 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc548ac-6d2c-48d0-abb4-9cfcba3fa3f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:14.234734 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc548ac-6d2c-48d0-abb4-9cfcba3fa3f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:14.236247 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc548ac-6d2c-48d0-abb4-9cfcba3fa3f5 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:14.250405 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 273/533] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:15:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:14.251191 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc548ac-6d2c-48d0-abb4-9cfcba3fa3f5 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:14.262706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75edd0bf-d4b7-47e0-97ae-563d0f6cb117 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:14.267656 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:14.310796 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0414a-656e-4eeb-9c75-d7a963403e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:14.311970 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0414a-656e-4eeb-9c75-d7a963403e1e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:14.311970 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0414a-656e-4eeb-9c75-d7a963403e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:14.312266 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0414a-656e-4eeb-9c75-d7a963403e1e None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:14.313044 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a0414a-656e-4eeb-9c75-d7a963403e1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:14.313363 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a0414a-656e-4eeb-9c75-d7a963403e1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:14.314748 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a0414a-656e-4eeb-9c75-d7a963403e1e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:14.328017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 261/534] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:14.328957 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0414a-656e-4eeb-9c75-d7a963403e1e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:14.376269 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc548ac-6d2c-48d0-abb4-9cfcba3fa3f5 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:14.416811 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:14.422519 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3475f-fb6f-411a-867d-c824923fbf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:14.423481 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3475f-fb6f-411a-867d-c824923fbf47 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:14.423865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3475f-fb6f-411a-867d-c824923fbf47 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:14.424244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3475f-fb6f-411a-867d-c824923fbf47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:14.434443 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea3475f-fb6f-411a-867d-c824923fbf47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:14.437163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea3475f-fb6f-411a-867d-c824923fbf47 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:14.453621 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0414a-656e-4eeb-9c75-d7a963403e1e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:14.456649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-0ea3475f-fb6f-411a-867d-c824923fbf47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:14.456649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:14.456649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:14.456649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:14.456649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:14.456649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:14.456649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:14.456649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:14.456649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:14.456649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:14.456649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:14.456649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:14.456649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:15:14.456649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:15:14.456649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:15:14.457526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:15:14.457526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:14.457526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:14.457526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:14.457526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:14.457526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:14.457526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:14.457526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:14.457526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:14.457526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:14.457526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:14.457526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:14.457526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:14.457526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:14.457526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:14.457526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:14.457526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:14.458346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:15:14.458346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:15:14.458346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:15:14.458346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:15:14.458346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:15:14.458346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:15:14.458346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:14.458346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:14.466936 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 274/535] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:14.467942 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3475f-fb6f-411a-867d-c824923fbf47 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:14.502247 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1916a3-cc11-4a30-b882-f141c477fa8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:14.503352 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1916a3-cc11-4a30-b882-f141c477fa8d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:14.503940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1916a3-cc11-4a30-b882-f141c477fa8d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:14.504392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1916a3-cc11-4a30-b882-f141c477fa8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:14.505391 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1916a3-cc11-4a30-b882-f141c477fa8d None admin] RBAC: Adding query filter params (role.id=8e315ffd561a48f0891ef53b51d548d8, scope.project.id=6ceffdd955324aec84ba048d2137ae00, user.id=95452d1b892a463d8e9e3eb16ca7c5ca) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:14.505919 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1916a3-cc11-4a30-b882-f141c477fa8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:14.507903 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1916a3-cc11-4a30-b882-f141c477fa8d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:14.514884 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 262/536] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:15:14 2022] GET /identity/v3/role_assignments?user.id=95452d1b892a463d8e9e3eb16ca7c5ca&scope.project.id=6ceffdd955324aec84ba048d2137ae00&role.id=8e315ffd561a48f0891ef53b51d548d8 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:14.515947 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1916a3-cc11-4a30-b882-f141c477fa8d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:14.580657 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:14.633612 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3475f-fb6f-411a-867d-c824923fbf47 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:14.702792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1916a3-cc11-4a30-b882-f141c477fa8d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:14.717693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a7355-18c7-4c8d-8c91-dff66c019f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:14.718494 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a7355-18c7-4c8d-8c91-dff66c019f8e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:14.718765 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a7355-18c7-4c8d-8c91-dff66c019f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:14.719066 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a7355-18c7-4c8d-8c91-dff66c019f8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:14.727294 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a7355-18c7-4c8d-8c91-dff66c019f8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:14.729093 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a7355-18c7-4c8d-8c91-dff66c019f8e None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:14.743534 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-851a7355-18c7-4c8d-8c91-dff66c019f8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:14.743534 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:14.743534 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:14.743534 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:14.743534 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:14.743534 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:14.743534 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:14.743534 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:14.743534 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:14.743534 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:14.743534 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:14.743534 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:14.743534 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:15:14.743534 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:15:14.743534 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:15:14.744996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:15:14.744996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:14.744996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:14.744996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:14.744996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:14.744996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:14.744996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:14.744996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:14.744996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:14.744996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:14.744996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:14.744996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:14.744996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:14.744996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:14.744996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:14.744996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:14.744996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:14.745898 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:15:14.745898 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:15:14.745898 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:15:14.745898 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:15:14.745898 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:15:14.745898 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:15:14.745898 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:14.745898 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:14.745898 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 275/537] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:14.752988 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a7355-18c7-4c8d-8c91-dff66c019f8e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:14.804479 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd65351-0e0b-4c47-90c5-410b030b47c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:14.805167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd65351-0e0b-4c47-90c5-410b030b47c4 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:14.805455 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd65351-0e0b-4c47-90c5-410b030b47c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:14.805733 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd65351-0e0b-4c47-90c5-410b030b47c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:14.806535 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd65351-0e0b-4c47-90c5-410b030b47c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:14.806860 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd65351-0e0b-4c47-90c5-410b030b47c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:14.808843 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd65351-0e0b-4c47-90c5-410b030b47c4 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:14.812248 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:14.854543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 263/538] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:15:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:14.857973 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd65351-0e0b-4c47-90c5-410b030b47c4 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:14.905180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a7355-18c7-4c8d-8c91-dff66c019f8e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:14.978780 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83114403-721b-458d-bd69-b896e9457724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:14.979566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83114403-721b-458d-bd69-b896e9457724 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:14.979912 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83114403-721b-458d-bd69-b896e9457724 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:14.980317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83114403-721b-458d-bd69-b896e9457724 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:14.981220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83114403-721b-458d-bd69-b896e9457724 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:14.982377 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83114403-721b-458d-bd69-b896e9457724 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:14.983902 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83114403-721b-458d-bd69-b896e9457724 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:15.002471 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:15.005684 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 276/539] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:15:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:15.012359 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83114403-721b-458d-bd69-b896e9457724 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:15.053699 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd65351-0e0b-4c47-90c5-410b030b47c4 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:15.103029 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8abda44-325f-41e5-96b0-d1347515f709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:15.103029 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abda44-325f-41e5-96b0-d1347515f709 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:15.103029 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abda44-325f-41e5-96b0-d1347515f709 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:15.103029 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abda44-325f-41e5-96b0-d1347515f709 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:15.103498 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8abda44-325f-41e5-96b0-d1347515f709 None admin] RBAC: Adding query filter params (role.id=8e315ffd561a48f0891ef53b51d548d8, scope.project.id=6ceffdd955324aec84ba048d2137ae00, user.id=c81ae2718faf4e59b799add42fb41c4f) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:15.103498 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8abda44-325f-41e5-96b0-d1347515f709 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:15.103498 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8abda44-325f-41e5-96b0-d1347515f709 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:15.108182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 264/540] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:15:14 2022] GET /identity/v3/role_assignments?user.id=c81ae2718faf4e59b799add42fb41c4f&scope.project.id=6ceffdd955324aec84ba048d2137ae00&role.id=8e315ffd561a48f0891ef53b51d548d8 => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:15.165521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83114403-721b-458d-bd69-b896e9457724 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:15.181779 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:15.204888 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa57cc9-ac27-4196-b827-8597c395e1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:15.205875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa57cc9-ac27-4196-b827-8597c395e1d3 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:15.206301 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa57cc9-ac27-4196-b827-8597c395e1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:15.206753 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa57cc9-ac27-4196-b827-8597c395e1d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:15.207814 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa57cc9-ac27-4196-b827-8597c395e1d3 None admin] RBAC: Adding query filter params (role.id=8e315ffd561a48f0891ef53b51d548d8, scope.project.id=6ceffdd955324aec84ba048d2137ae00, user.id=1af0160cd3da4a0bb391b81f083715d9) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:15.208290 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa57cc9-ac27-4196-b827-8597c395e1d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:15.209948 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa57cc9-ac27-4196-b827-8597c395e1d3 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:15.221472 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 277/541] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:15:15 2022] GET /identity/v3/role_assignments?user.id=1af0160cd3da4a0bb391b81f083715d9&scope.project.id=6ceffdd955324aec84ba048d2137ae00&role.id=8e315ffd561a48f0891ef53b51d548d8 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:15.303547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622c84b-e257-49d0-99d0-8cbb590bd870 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:15.304072 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622c84b-e257-49d0-99d0-8cbb590bd870 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:15.304072 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622c84b-e257-49d0-99d0-8cbb590bd870 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:15.305919 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 265/542] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:15.311903 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c99e1-55ca-45f4-bef7-06a7a24b7d1e None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:15.312355 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e9fe0-a8e7-4755-87a1-a6ae948d78c9 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:15.312689 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c99e1-55ca-45f4-bef7-06a7a24b7d1e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:15.312791 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e9fe0-a8e7-4755-87a1-a6ae948d78c9 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:15.313056 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e9fe0-a8e7-4755-87a1-a6ae948d78c9 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:15.313395 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c99e1-55ca-45f4-bef7-06a7a24b7d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:15.314847 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 266/543] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:15.320488 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc4d28-168b-4b49-9341-8ebdac6ae338 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:15.320775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc4d28-168b-4b49-9341-8ebdac6ae338 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:15.321047 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc4d28-168b-4b49-9341-8ebdac6ae338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:15.334298 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-901c99e1-55ca-45f4-bef7-06a7a24b7d1e None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:15.335942 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-901c99e1-55ca-45f4-bef7-06a7a24b7d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:15.352111 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-5cbc4d28-168b-4b49-9341-8ebdac6ae338 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:15.353370 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-5cbc4d28-168b-4b49-9341-8ebdac6ae338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:15.431305 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 278/544] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:15:15.459110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c99e1-55ca-45f4-bef7-06a7a24b7d1e None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:15.476954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 267/545] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:15:15.511699 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc4d28-168b-4b49-9341-8ebdac6ae338 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:15.625771 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c99e1-55ca-45f4-bef7-06a7a24b7d1e None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:15.638204 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:15.647978 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:15.670224 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2d7a5-929e-422e-8399-f075f444d8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:15.671050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2d7a5-929e-422e-8399-f075f444d8ca None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:15.671328 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2d7a5-929e-422e-8399-f075f444d8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:15.671597 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2d7a5-929e-422e-8399-f075f444d8ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:15.680471 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc4d28-168b-4b49-9341-8ebdac6ae338 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:15.695331 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e2d7a5-929e-422e-8399-f075f444d8ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:15.705615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-35e2d7a5-929e-422e-8399-f075f444d8ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:15.705615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:15.705615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:15.705615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:15.705615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:15.705615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:15.705615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:15.705615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:15.705615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:15.705615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:15.705615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:15.705615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:15.705615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:15.705615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:15.705615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:15.705615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:15.706492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:15.706492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:15.706492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:15.706492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:15.706492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:15.706492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:15.706492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:15.706492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:15.706492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:15.706492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:15.706492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:15.706492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:15.706492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:15.706492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:15.706492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:15.706492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:15.707787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:15.707787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:15.707787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:15.707787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:15.707787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:15.707787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:15.707787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:15.707787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:15.707787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:15.707787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:15.707787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:15.707787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:15.707787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 279/546] 10.222.0.24 () {56 vars in 1102 bytes} [Fri May 27 03:15:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:15:15.719902 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2d7a5-929e-422e-8399-f075f444d8ca None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:15.736638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11985601-7ae7-4afa-8b33-f046e2191715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:15.738028 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11985601-7ae7-4afa-8b33-f046e2191715 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:15.738616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11985601-7ae7-4afa-8b33-f046e2191715 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:15.739361 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11985601-7ae7-4afa-8b33-f046e2191715 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:15.763919 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11985601-7ae7-4afa-8b33-f046e2191715 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:15.772690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-11985601-7ae7-4afa-8b33-f046e2191715 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:15.772690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:15.772690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:15.772690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:15.772690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:15.772690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:15.772690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:15.772690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:15.772690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:15.772690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:15.772690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:15.772690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:15.772690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:15.772690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:15.772690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:15.772690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:15.773556 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:15.773556 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:15.773556 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:15.773556 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:15.773556 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:15.773556 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:15.773556 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:15.773556 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:15.773556 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:15.773556 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:15.773556 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:15.773556 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:15.773556 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:15.773556 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:15.773556 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:15.773556 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:15.774746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:15.774746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:15.774746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:15.774746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:15.774746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:15.774746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:15.774746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:15.774746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:15.774746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:15.774746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:15.774746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:15.774746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:15.774746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 268/547] 10.222.0.24 () {56 vars in 1102 bytes} [Fri May 27 03:15:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:15:15.786776 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11985601-7ae7-4afa-8b33-f046e2191715 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:15.908386 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2d7a5-929e-422e-8399-f075f444d8ca None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:15.964058 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11985601-7ae7-4afa-8b33-f046e2191715 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:15.971558 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455147b5-58ea-453b-bee7-0019592ab558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:15.972239 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455147b5-58ea-453b-bee7-0019592ab558 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:15.972525 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455147b5-58ea-453b-bee7-0019592ab558 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:15.972800 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455147b5-58ea-453b-bee7-0019592ab558 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:15.973559 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455147b5-58ea-453b-bee7-0019592ab558 None admin] RBAC: Adding query filter params (name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:15.973861 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455147b5-58ea-453b-bee7-0019592ab558 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:15.979724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455147b5-58ea-453b-bee7-0019592ab558 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:15.994189 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 280/548] 10.222.0.24 () {56 vars in 1103 bytes} [Fri May 27 03:15:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:16.000021 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455147b5-58ea-453b-bee7-0019592ab558 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:16.003927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4050fdb-e0ef-4835-8958-0df279a05c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:16.004838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4050fdb-e0ef-4835-8958-0df279a05c64 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:16.005387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4050fdb-e0ef-4835-8958-0df279a05c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:16.005839 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4050fdb-e0ef-4835-8958-0df279a05c64 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:16.006870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4050fdb-e0ef-4835-8958-0df279a05c64 None admin] RBAC: Adding query filter params (name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:16.007378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4050fdb-e0ef-4835-8958-0df279a05c64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:16.008946 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4050fdb-e0ef-4835-8958-0df279a05c64 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:16.018464 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 269/549] 10.222.0.24 () {56 vars in 1103 bytes} [Fri May 27 03:15:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:16.024079 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4050fdb-e0ef-4835-8958-0df279a05c64 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:16.033573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:16.060598 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:16.135709 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4050fdb-e0ef-4835-8958-0df279a05c64 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:16.171696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c7987-5b5f-44e2-bfd7-8739c68c04cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:16.172793 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c7987-5b5f-44e2-bfd7-8739c68c04cd None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:16.173257 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c7987-5b5f-44e2-bfd7-8739c68c04cd None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:16.173747 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c7987-5b5f-44e2-bfd7-8739c68c04cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:16.184950 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c7987-5b5f-44e2-bfd7-8739c68c04cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:16.186658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c7987-5b5f-44e2-bfd7-8739c68c04cd None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:16.205384 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-e32c7987-5b5f-44e2-bfd7-8739c68c04cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:16.205384 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:16.205384 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:16.205384 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:16.205384 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:16.205384 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:16.205384 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:16.205384 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:16.205384 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:16.205384 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:16.205384 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:16.205384 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:16.205384 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:16.205384 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:16.205384 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:16.206307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:16.206307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:16.206307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:16.206307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:16.206307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:16.206307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:16.206307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:16.206307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:16.206307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:16.206307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:16.206307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:16.206307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:16.206307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:16.206307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:16.206307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:16.206307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:16.206307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:16.207392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:16.207392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:16.207392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:16.207392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:16.209254 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 270/550] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:15:16.215413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c7987-5b5f-44e2-bfd7-8739c68c04cd None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:16.254002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455147b5-58ea-453b-bee7-0019592ab558 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:16.313553 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff4c0b-827d-4657-a301-538fc8baf6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:16.314098 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:16.314539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff4c0b-827d-4657-a301-538fc8baf6a8 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:16.315711 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff4c0b-827d-4657-a301-538fc8baf6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:16.315994 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff4c0b-827d-4657-a301-538fc8baf6a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:16.336851 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff4c0b-827d-4657-a301-538fc8baf6a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:16.338868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff4c0b-827d-4657-a301-538fc8baf6a8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:16.367482 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-3aff4c0b-827d-4657-a301-538fc8baf6a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:16.367482 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:16.367482 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:16.367482 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:16.367482 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:16.367482 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:16.367482 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:16.367482 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:16.367482 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:16.367482 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:16.367482 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:16.367482 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:16.367482 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:16.367482 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:16.367482 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:16.368582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:16.368582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:16.368582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:16.368582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:16.368582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:16.368582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:16.368582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:16.368582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:16.368582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:16.368582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:16.368582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:16.368582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:16.368582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:16.368582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:16.368582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:16.368582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:16.368582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:16.370112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:16.370112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:16.370112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:16.370112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:16.371724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 281/551] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:16.383768 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff4c0b-827d-4657-a301-538fc8baf6a8 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:16.387913 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c7987-5b5f-44e2-bfd7-8739c68c04cd None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:16.444757 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a270e5d-e0cb-4442-a2f4-00ad0a93a13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:16.445419 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a270e5d-e0cb-4442-a2f4-00ad0a93a13d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:16.445696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a270e5d-e0cb-4442-a2f4-00ad0a93a13d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:16.445968 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a270e5d-e0cb-4442-a2f4-00ad0a93a13d None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:16.446693 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a270e5d-e0cb-4442-a2f4-00ad0a93a13d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:16.447024 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a270e5d-e0cb-4442-a2f4-00ad0a93a13d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:16.448972 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a270e5d-e0cb-4442-a2f4-00ad0a93a13d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:16.475554 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 271/552] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:16.479200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a270e5d-e0cb-4442-a2f4-00ad0a93a13d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:16.528625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:16.551836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff4c0b-827d-4657-a301-538fc8baf6a8 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:16.620113 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f1b8d-ecef-46ee-97b1-39529ec0dfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:16.621922 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f1b8d-ecef-46ee-97b1-39529ec0dfb3 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:16.622203 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f1b8d-ecef-46ee-97b1-39529ec0dfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:16.622473 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f1b8d-ecef-46ee-97b1-39529ec0dfb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:16.623214 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f1b8d-ecef-46ee-97b1-39529ec0dfb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:16.623516 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f1b8d-ecef-46ee-97b1-39529ec0dfb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:16.631888 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f1b8d-ecef-46ee-97b1-39529ec0dfb3 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:16.654330 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 282/553] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:16.659290 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f1b8d-ecef-46ee-97b1-39529ec0dfb3 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:16.660620 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a270e5d-e0cb-4442-a2f4-00ad0a93a13d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:16.678613 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:16.696751 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31494e22-3cc2-4299-99a1-6b21d4102702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:16.698071 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31494e22-3cc2-4299-99a1-6b21d4102702 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:16.698352 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31494e22-3cc2-4299-99a1-6b21d4102702 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:16.698629 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31494e22-3cc2-4299-99a1-6b21d4102702 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:16.706531 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31494e22-3cc2-4299-99a1-6b21d4102702 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:16.708088 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31494e22-3cc2-4299-99a1-6b21d4102702 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:16.724041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-31494e22-3cc2-4299-99a1-6b21d4102702 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 03:15:16.724041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:16.724041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:16.724041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:16.724041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:16.724041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:16.724041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:16.724041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:16.724041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:16.724041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:16.724041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:16.724041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:16.724041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:15:16.724041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:15:16.724041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:15:16.724041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:15:16.724983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:16.724983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:16.724983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:15:16.724983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:16.724983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:15:16.724983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:16.724983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:16.724983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:16.724983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:16.724983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:16.724983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:16.724983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:16.724983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:16.724983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:16.724983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:16.724983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:16.724983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:16.724983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:16.725928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:16.725928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:16.725928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:15:16.725928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:15:16.725928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:15:16.725928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:15:16.725928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:15:16.725928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:15:16.725928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 03:15:16.725928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:16.725928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 272/554] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:15:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:16.733792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31494e22-3cc2-4299-99a1-6b21d4102702 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:16.845931 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31494e22-3cc2-4299-99a1-6b21d4102702 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:16.880605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f7e21-2b4b-46c5-9ce5-3cdc2e4684e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:16.881274 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f7e21-2b4b-46c5-9ce5-3cdc2e4684e1 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:16.881552 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f7e21-2b4b-46c5-9ce5-3cdc2e4684e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:16.881820 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f7e21-2b4b-46c5-9ce5-3cdc2e4684e1 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:16.882651 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f7e21-2b4b-46c5-9ce5-3cdc2e4684e1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:16.882985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f7e21-2b4b-46c5-9ce5-3cdc2e4684e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:16.884413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f7e21-2b4b-46c5-9ce5-3cdc2e4684e1 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:16.902972 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 273/555] 10.222.0.24 () {56 vars in 1135 bytes} [Fri May 27 03:15:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:16.908851 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f7e21-2b4b-46c5-9ce5-3cdc2e4684e1 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:16.920902 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f1b8d-ecef-46ee-97b1-39529ec0dfb3 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:16.983349 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:16.989576 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb0bba2-9a78-4141-88d2-88660db980ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:16.990486 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0bba2-9a78-4141-88d2-88660db980ac None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:16.990765 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0bba2-9a78-4141-88d2-88660db980ac None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:16.991057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb0bba2-9a78-4141-88d2-88660db980ac None admin] PATH_INFO: `/v3/users/nova` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:16.999658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb0bba2-9a78-4141-88d2-88660db980ac None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:17.001151 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb0bba2-9a78-4141-88d2-88660db980ac None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:17.003711 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:17.017321 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-deb0bba2-9a78-4141-88d2-88660db980ac None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:15:17.017321 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:17.017321 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:17.017321 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:17.017321 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:17.017321 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:17.017321 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:17.017321 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:17.017321 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:17.017321 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:17.017321 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:17.017321 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:17.017321 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:15:17.017321 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:15:17.017321 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:15:17.017321 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:15:17.023144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:17.023144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:17.023144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:15:17.023144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:17.023144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:15:17.023144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:17.023144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:17.023144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:17.023144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:17.023144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:17.023144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:17.023144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:17.023144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:17.023144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:17.023144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:17.023144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:17.023144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:17.023144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:17.024510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:17.024510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:17.024510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:15:17.024510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:15:17.024510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:15:17.024510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:15:17.024510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:15:17.024510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:15:17.024510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:15:17.024510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:17.024510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 283/556] 10.222.0.24 () {56 vars in 1093 bytes} [Fri May 27 03:15:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:15:17.024510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb0bba2-9a78-4141-88d2-88660db980ac None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:17.158964 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f7e21-2b4b-46c5-9ce5-3cdc2e4684e1 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:17.183927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb0bba2-9a78-4141-88d2-88660db980ac None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:17.203269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e35367-4d29-4488-a115-14459cff8962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:17.203922 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e35367-4d29-4488-a115-14459cff8962 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:17.204203 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e35367-4d29-4488-a115-14459cff8962 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:17.204476 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e35367-4d29-4488-a115-14459cff8962 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:17.220191 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee49d0-6bb9-4412-9789-497a86981189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:17.220860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee49d0-6bb9-4412-9789-497a86981189 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:17.221141 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee49d0-6bb9-4412-9789-497a86981189 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:17.221421 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee49d0-6bb9-4412-9789-497a86981189 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:17.222260 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee49d0-6bb9-4412-9789-497a86981189 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:17.222564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee49d0-6bb9-4412-9789-497a86981189 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:17.224070 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee49d0-6bb9-4412-9789-497a86981189 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:17.262523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 284/557] 10.222.0.24 () {56 vars in 1133 bytes} [Fri May 27 03:15:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:17.263882 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953c4d7-d20c-484d-b64c-f56de5a50b58 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:17.264179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953c4d7-d20c-484d-b64c-f56de5a50b58 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:17.264454 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953c4d7-d20c-484d-b64c-f56de5a50b58 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:17.269805 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 285/558] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:17.270617 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953c4d7-d20c-484d-b64c-f56de5a50b58 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:17.288920 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e35367-4d29-4488-a115-14459cff8962 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:17.290754 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e35367-4d29-4488-a115-14459cff8962 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:17.333251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:17.346693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:17.392956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953c4d7-d20c-484d-b64c-f56de5a50b58 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:17.402633 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 274/559] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:15:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:15:17.404093 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92b122-39e1-4397-a908-b978c7d3413a None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:17.404460 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92b122-39e1-4397-a908-b978c7d3413a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:17.404739 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad92b122-39e1-4397-a908-b978c7d3413a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:17.422560 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-ad92b122-39e1-4397-a908-b978c7d3413a None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:17.423777 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ad92b122-39e1-4397-a908-b978c7d3413a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:17.430585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f477e59-48fc-430e-98f2-fb6f9a3fad26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:17.431432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f477e59-48fc-430e-98f2-fb6f9a3fad26 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:17.431896 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f477e59-48fc-430e-98f2-fb6f9a3fad26 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:17.432170 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f477e59-48fc-430e-98f2-fb6f9a3fad26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:17.443758 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f477e59-48fc-430e-98f2-fb6f9a3fad26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:17.445296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f477e59-48fc-430e-98f2-fb6f9a3fad26 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:17.465033 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-0f477e59-48fc-430e-98f2-fb6f9a3fad26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:17.465033 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:17.465033 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:17.465033 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:17.465033 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:17.465033 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:17.465033 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:17.465033 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:17.465033 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:17.465033 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:17.465033 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:17.465033 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:17.465033 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:17.465033 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:17.465033 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:17.467157 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:17.467157 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:17.467157 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:17.467157 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:17.467157 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:17.467157 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:17.467157 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:17.467157 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:17.467157 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:17.467157 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:17.467157 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:17.467157 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:17.467157 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:17.467157 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:17.467157 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:17.467157 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:17.467157 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:17.469215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:17.469215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:17.469215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:17.469215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:17.469215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 286/560] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:17.469215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f477e59-48fc-430e-98f2-fb6f9a3fad26 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:17.492869 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 275/561] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:15:17.499678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92b122-39e1-4397-a908-b978c7d3413a None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:17.581964 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:17.584457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f477e59-48fc-430e-98f2-fb6f9a3fad26 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:17.595436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:17.620198 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a67740-3824-4110-9f04-db769524d20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:17.620986 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a67740-3824-4110-9f04-db769524d20f None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:17.621298 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a67740-3824-4110-9f04-db769524d20f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:17.621558 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad92b122-39e1-4397-a908-b978c7d3413a None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:17.621721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a67740-3824-4110-9f04-db769524d20f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:17.637162 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a67740-3824-4110-9f04-db769524d20f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:17.639050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a67740-3824-4110-9f04-db769524d20f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:17.660721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-a8a67740-3824-4110-9f04-db769524d20f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:17.660721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:17.660721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:17.660721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:17.660721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:17.660721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:17.660721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:17.660721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:17.660721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:17.660721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:17.660721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:17.660721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:17.660721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:17.660721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:17.660721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:17.662646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:17.662646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:17.662646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:17.662646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:17.662646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:17.662646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:17.662646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:17.662646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:17.662646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:17.662646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:17.662646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:17.662646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:17.662646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:17.662646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:17.662646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:17.662646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:17.662646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:17.664422 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:17.664422 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:17.664422 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:17.664422 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:17.664869 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15232fa7-1899-408f-b794-65d2cca77785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:17.665018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 287/562] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:17.665181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15232fa7-1899-408f-b794-65d2cca77785 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:17.665181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15232fa7-1899-408f-b794-65d2cca77785 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:17.665181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15232fa7-1899-408f-b794-65d2cca77785 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:17.665545 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a67740-3824-4110-9f04-db769524d20f None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:17.665674 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15232fa7-1899-408f-b794-65d2cca77785 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:17.665674 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15232fa7-1899-408f-b794-65d2cca77785 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:17.674873 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15232fa7-1899-408f-b794-65d2cca77785 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:17.680722 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 276/563] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:17.681650 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15232fa7-1899-408f-b794-65d2cca77785 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:17.770086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a67740-3824-4110-9f04-db769524d20f None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:17.786132 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15232fa7-1899-408f-b794-65d2cca77785 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:17.804553 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a07d0-4943-4bb1-ad4b-89f4e0b3d667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:17.805350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a07d0-4943-4bb1-ad4b-89f4e0b3d667 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:17.805631 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a07d0-4943-4bb1-ad4b-89f4e0b3d667 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:17.805904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a07d0-4943-4bb1-ad4b-89f4e0b3d667 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:17.811838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:17.817145 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a07d0-4943-4bb1-ad4b-89f4e0b3d667 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:17.823604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a001aebb-9982-48df-98d1-f72d418de1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:17.826191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001aebb-9982-48df-98d1-f72d418de1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:17.826191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001aebb-9982-48df-98d1-f72d418de1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:17.826191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001aebb-9982-48df-98d1-f72d418de1a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:17.826407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-da1a07d0-4943-4bb1-ad4b-89f4e0b3d667 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:17.826407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:17.826407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:17.826407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:17.826407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:17.826407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:17.826407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:17.826407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:17.826407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:17.826407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:17.826407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:17.826407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:17.826407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:17.826407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:17.826407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:17.826407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:17.832212 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:17.832212 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:17.832212 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:17.832212 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:17.832212 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:17.832212 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:17.832212 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:17.832212 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:17.832212 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:17.832212 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:17.832212 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:17.832212 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:17.832212 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:17.832212 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:17.832212 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:17.832212 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:17.833283 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:17.833283 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:17.833283 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:17.833283 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:17.833283 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:17.833283 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:17.833283 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:17.833283 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:17.833283 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:17.833283 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:17.833283 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:17.833283 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:17.834067 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a001aebb-9982-48df-98d1-f72d418de1a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:17.834067 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a001aebb-9982-48df-98d1-f72d418de1a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:17.834224 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 288/564] 10.222.0.24 () {56 vars in 1102 bytes} [Fri May 27 03:15:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:17.834318 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a001aebb-9982-48df-98d1-f72d418de1a5 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:17.834412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a07d0-4943-4bb1-ad4b-89f4e0b3d667 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:17.846713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 277/565] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:17.847142 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bc1b2-cef7-4be9-ab9f-b053efe135f5 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:17.847430 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bc1b2-cef7-4be9-ab9f-b053efe135f5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:17.847707 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bc1b2-cef7-4be9-ab9f-b053efe135f5 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:17.849188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 278/566] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:17.849897 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29bc1b2-cef7-4be9-ab9f-b053efe135f5 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:17.858648 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:17.931369 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a07d0-4943-4bb1-ad4b-89f4e0b3d667 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:17.951162 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29bc1b2-cef7-4be9-ab9f-b053efe135f5 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:17.965574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3656048f-6bf0-45c8-b3f1-eaa9c7eb123a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:17.966354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656048f-6bf0-45c8-b3f1-eaa9c7eb123a None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:17.966665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656048f-6bf0-45c8-b3f1-eaa9c7eb123a None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:17.966989 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656048f-6bf0-45c8-b3f1-eaa9c7eb123a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:17.973464 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3656048f-6bf0-45c8-b3f1-eaa9c7eb123a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:17.975153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3656048f-6bf0-45c8-b3f1-eaa9c7eb123a None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:17.983987 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603f4ce2-0f6a-492c-9320-a38bbc9f51ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:17.984642 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f4ce2-0f6a-492c-9320-a38bbc9f51ad None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:17.984922 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f4ce2-0f6a-492c-9320-a38bbc9f51ad None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:17.985192 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f4ce2-0f6a-492c-9320-a38bbc9f51ad None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:17.985935 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603f4ce2-0f6a-492c-9320-a38bbc9f51ad None admin] RBAC: Adding query filter params (name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:17.986242 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603f4ce2-0f6a-492c-9320-a38bbc9f51ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:17.987706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603f4ce2-0f6a-492c-9320-a38bbc9f51ad None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:17.988403 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-3656048f-6bf0-45c8-b3f1-eaa9c7eb123a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:17.988403 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:17.988403 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:17.988403 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:17.988403 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:17.988403 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:17.988403 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:17.988403 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:17.988403 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:17.988403 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:17.988403 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:17.988403 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:17.988403 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:15:17.988403 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:15:17.988403 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:15:17.989310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:15:17.989310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:17.989310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:17.989310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:17.989310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:17.989310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:17.989310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:17.989310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:17.989310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:17.989310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:17.989310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:17.989310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:17.989310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:17.989310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:17.989310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:17.989310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:17.989310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:17.990200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:15:17.990200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:15:17.990200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:15:17.990200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:15:17.990200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:15:17.990200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:15:17.990200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:17.990200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:17.990200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 289/567] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:17.990717 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6969f-b8c7-4e45-bc61-e941c6d5b8db None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:17.991046 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6969f-b8c7-4e45-bc61-e941c6d5b8db None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:17.991318 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6969f-b8c7-4e45-bc61-e941c6d5b8db None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:17.992550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 290/568] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:17.993268 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6969f-b8c7-4e45-bc61-e941c6d5b8db None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:18.005647 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 279/569] 10.222.0.24 () {56 vars in 1103 bytes} [Fri May 27 03:15:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:18.005647 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4edad4-0c22-4980-88b4-f88e0515f7ff None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:18.005647 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4edad4-0c22-4980-88b4-f88e0515f7ff None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:18.005647 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4edad4-0c22-4980-88b4-f88e0515f7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:18.020418 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-3e4edad4-0c22-4980-88b4-f88e0515f7ff None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:18.021597 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3e4edad4-0c22-4980-88b4-f88e0515f7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:18.062287 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:18.086897 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 280/570] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:15:18.087713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4edad4-0c22-4980-88b4-f88e0515f7ff None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:18.092562 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:18.111929 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6969f-b8c7-4e45-bc61-e941c6d5b8db None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:18.146425 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd6cbbe-2c2c-4138-9b8c-ad5e65f3ed6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:18.147246 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6cbbe-2c2c-4138-9b8c-ad5e65f3ed6a None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:18.147516 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6cbbe-2c2c-4138-9b8c-ad5e65f3ed6a None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:18.147786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6cbbe-2c2c-4138-9b8c-ad5e65f3ed6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:18.158465 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd6cbbe-2c2c-4138-9b8c-ad5e65f3ed6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:18.159950 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd6cbbe-2c2c-4138-9b8c-ad5e65f3ed6a None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:18.175095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-6bd6cbbe-2c2c-4138-9b8c-ad5e65f3ed6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:18.175095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:18.175095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:18.175095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:18.175095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:18.175095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:18.175095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:18.175095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:18.175095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:18.175095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:18.175095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:18.175095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:18.175095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:15:18.175095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:15:18.175095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:15:18.175990 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:15:18.175990 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:18.175990 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:18.175990 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:18.175990 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:18.175990 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:18.175990 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:18.175990 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:18.175990 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:18.175990 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:18.175990 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:18.175990 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:18.175990 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:18.175990 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:18.175990 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:18.175990 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:18.175990 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:18.176868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:15:18.176868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:15:18.176868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:15:18.176868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:15:18.176868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:15:18.176868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:15:18.176868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:18.176868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:18.176868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 291/571] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:18.177588 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091000b-3cc2-4976-8d4c-129ace1de0d4 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:18.177840 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091000b-3cc2-4976-8d4c-129ace1de0d4 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:18.178115 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9091000b-3cc2-4976-8d4c-129ace1de0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:18.194604 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-9091000b-3cc2-4976-8d4c-129ace1de0d4 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:18.195889 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-9091000b-3cc2-4976-8d4c-129ace1de0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:18.197179 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4edad4-0c22-4980-88b4-f88e0515f7ff None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:18.231197 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7488429-976a-484a-9993-6c2e166defec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:18.231847 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7488429-976a-484a-9993-6c2e166defec None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:18.232126 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7488429-976a-484a-9993-6c2e166defec None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:18.232398 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7488429-976a-484a-9993-6c2e166defec None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:18.233181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7488429-976a-484a-9993-6c2e166defec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:18.233489 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7488429-976a-484a-9993-6c2e166defec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:18.234937 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7488429-976a-484a-9993-6c2e166defec None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:18.246615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 281/572] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:15:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:18.247440 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7488429-976a-484a-9993-6c2e166defec None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:18.266011 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 292/573] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:15:18.266860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091000b-3cc2-4976-8d4c-129ace1de0d4 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:18.292742 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:18.335356 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:18.357108 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7488429-976a-484a-9993-6c2e166defec None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:18.363381 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9091000b-3cc2-4976-8d4c-129ace1de0d4 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:18.391102 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbcbd98-6071-4897-ba1c-c295c904bfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:18.391893 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcbd98-6071-4897-ba1c-c295c904bfc9 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:18.392175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcbd98-6071-4897-ba1c-c295c904bfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:18.392448 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcbd98-6071-4897-ba1c-c295c904bfc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:18.397802 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c00d6d-576b-4f7f-b9aa-f372cd6cf730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:18.398580 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c00d6d-576b-4f7f-b9aa-f372cd6cf730 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:18.398885 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c00d6d-576b-4f7f-b9aa-f372cd6cf730 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:18.399164 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c00d6d-576b-4f7f-b9aa-f372cd6cf730 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:18.402534 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbcbd98-6071-4897-ba1c-c295c904bfc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:18.403999 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbcbd98-6071-4897-ba1c-c295c904bfc9 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:18.409041 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c00d6d-576b-4f7f-b9aa-f372cd6cf730 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:18.417121 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-05c00d6d-576b-4f7f-b9aa-f372cd6cf730 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:18.417121 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:18.417121 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:18.417121 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:18.417121 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:18.417121 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:18.417121 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:18.417121 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:18.417121 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:18.417121 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:18.417121 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:18.417121 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:18.417121 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:18.417121 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:18.417121 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:18.417121 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:18.418071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:18.418071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:18.418071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:18.418071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:18.418071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:18.418071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:18.418071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:18.418071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:18.418071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:18.418071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:18.418071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:18.418071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:18.418071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:18.418071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:18.418071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:18.418071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:18.419227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:18.419227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:18.419227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:18.419227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:18.419227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:18.419227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:18.419227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:18.419227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:18.419227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:18.419227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:18.419227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:18.419227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:18.419227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 293/574] 10.222.0.24 () {56 vars in 1102 bytes} [Fri May 27 03:15:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:18.419227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c00d6d-576b-4f7f-b9aa-f372cd6cf730 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:18.421570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-0bbcbd98-6071-4897-ba1c-c295c904bfc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:18.421570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:18.421570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:18.421570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:18.421570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:18.421570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:18.421570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:18.421570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:18.421570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:18.421570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:18.421570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:18.421570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:18.421570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:18.421570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:18.421570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:18.422548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:18.422548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:18.422548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:18.422548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:18.422548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:18.422548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:18.422548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:18.422548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:18.422548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:18.422548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:18.422548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:18.422548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:18.422548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:18.422548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:18.422548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:18.422548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:18.422548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:18.424275 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:18.424275 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:18.424275 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:18.424275 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:18.424275 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 282/575] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:18.424275 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbcbd98-6071-4897-ba1c-c295c904bfc9 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:18.523050 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:18.526927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c00d6d-576b-4f7f-b9aa-f372cd6cf730 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:18.535002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbcbd98-6071-4897-ba1c-c295c904bfc9 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:18.555550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:18.560178 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08621704-40a3-4c3c-9f00-919884a698ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:18.560833 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08621704-40a3-4c3c-9f00-919884a698ff None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:18.561117 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08621704-40a3-4c3c-9f00-919884a698ff None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:18.561418 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08621704-40a3-4c3c-9f00-919884a698ff None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:18.562169 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08621704-40a3-4c3c-9f00-919884a698ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:18.562474 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08621704-40a3-4c3c-9f00-919884a698ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:18.564015 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08621704-40a3-4c3c-9f00-919884a698ff None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:18.568562 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b8d40-7f06-41b7-a689-fd755a20d81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:18.569443 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b8d40-7f06-41b7-a689-fd755a20d81a None admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:18.569720 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b8d40-7f06-41b7-a689-fd755a20d81a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:18.569991 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b8d40-7f06-41b7-a689-fd755a20d81a None admin] PATH_INFO: `/v3/projects/6ceffdd955324aec84ba048d2137ae00/users/22a9592143014f3287cd75c60ca17237/roles/8e315ffd561a48f0891ef53b51d548d8` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:18.575958 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 294/576] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:15:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:18.576753 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08621704-40a3-4c3c-9f00-919884a698ff None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:18.587592 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b8d40-7f06-41b7-a689-fd755a20d81a None admin] RBAC: Authorizing `identity:create_grant(project_id=6ceffdd955324aec84ba048d2137ae00, user_id=22a9592143014f3287cd75c60ca17237, role_id=8e315ffd561a48f0891ef53b51d548d8)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:18.589604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b8d40-7f06-41b7-a689-fd755a20d81a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:18.673059 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08621704-40a3-4c3c-9f00-919884a698ff None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:18.677100 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 283/577] 10.222.0.24 () {58 vars in 1439 bytes} [Fri May 27 03:15:18 2022] PUT /identity/v3/projects/6ceffdd955324aec84ba048d2137ae00/users/22a9592143014f3287cd75c60ca17237/roles/8e315ffd561a48f0891ef53b51d548d8 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:15:18.677911 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b8d40-7f06-41b7-a689-fd755a20d81a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:18.720055 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b32206-b4e2-4c11-a968-1b477c7df97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:18.720867 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b32206-b4e2-4c11-a968-1b477c7df97e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:18.721150 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b32206-b4e2-4c11-a968-1b477c7df97e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:18.721424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b32206-b4e2-4c11-a968-1b477c7df97e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:18.734216 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b32206-b4e2-4c11-a968-1b477c7df97e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:18.742281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-c1b32206-b4e2-4c11-a968-1b477c7df97e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:18.742281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:18.742281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:18.742281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:18.742281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:18.742281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:18.742281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:18.742281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:18.742281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:18.742281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:18.742281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:18.742281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:18.742281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:18.742281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:18.742281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:18.742281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:18.743388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:18.743388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:18.743388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:18.743388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:18.743388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:18.743388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:18.743388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:18.743388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:18.743388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:18.743388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:18.743388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:18.743388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:18.743388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:18.743388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:18.743388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:18.743388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:18.744238 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:18.744238 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:18.744238 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:18.744238 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:18.744238 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:18.744238 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:18.744238 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:18.744238 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:18.744238 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:18.744238 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:18.744238 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:18.744238 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:18.744238 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 295/578] 10.222.0.24 () {56 vars in 1102 bytes} [Fri May 27 03:15:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:18.745831 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b32206-b4e2-4c11-a968-1b477c7df97e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:18.809143 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b8d40-7f06-41b7-a689-fd755a20d81a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:18.812102 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:18.851720 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e96db-fe70-49dc-85cf-da70ffa02830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:18.852401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e96db-fe70-49dc-85cf-da70ffa02830 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:18.852690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e96db-fe70-49dc-85cf-da70ffa02830 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:18.852967 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e96db-fe70-49dc-85cf-da70ffa02830 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:18.853792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e96db-fe70-49dc-85cf-da70ffa02830 None admin] RBAC: Adding query filter params (name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:18.854107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e96db-fe70-49dc-85cf-da70ffa02830 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:18.855648 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e96db-fe70-49dc-85cf-da70ffa02830 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:18.862397 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:18.866739 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 284/579] 10.222.0.24 () {56 vars in 1103 bytes} [Fri May 27 03:15:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:18.867913 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e96db-fe70-49dc-85cf-da70ffa02830 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:18.872943 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b32206-b4e2-4c11-a968-1b477c7df97e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:18.914892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31926e41-7f0c-482c-b49d-ee97667f51e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:18.915588 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31926e41-7f0c-482c-b49d-ee97667f51e5 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:18.915867 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31926e41-7f0c-482c-b49d-ee97667f51e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:18.916171 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31926e41-7f0c-482c-b49d-ee97667f51e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:18.916958 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31926e41-7f0c-482c-b49d-ee97667f51e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:18.917264 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31926e41-7f0c-482c-b49d-ee97667f51e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:18.918635 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31926e41-7f0c-482c-b49d-ee97667f51e5 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:18.931945 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 296/580] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:18.936295 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31926e41-7f0c-482c-b49d-ee97667f51e5 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:18.986589 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e96db-fe70-49dc-85cf-da70ffa02830 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:19.023357 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c971f-0379-41e2-a33d-5b0cf5464164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:19.024438 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c971f-0379-41e2-a33d-5b0cf5464164 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:19.024934 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c971f-0379-41e2-a33d-5b0cf5464164 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:19.025488 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c971f-0379-41e2-a33d-5b0cf5464164 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:19.026423 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c971f-0379-41e2-a33d-5b0cf5464164 None admin] RBAC: Adding query filter params (role.id=8e315ffd561a48f0891ef53b51d548d8, scope.project.id=6ceffdd955324aec84ba048d2137ae00, user.id=21d559cd7127490da84185f20684929d) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:19.026821 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c971f-0379-41e2-a33d-5b0cf5464164 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:19.028308 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c971f-0379-41e2-a33d-5b0cf5464164 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:19.038875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 285/581] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:15:18 2022] GET /identity/v3/role_assignments?user.id=21d559cd7127490da84185f20684929d&scope.project.id=6ceffdd955324aec84ba048d2137ae00&role.id=8e315ffd561a48f0891ef53b51d548d8 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:19.038875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c971f-0379-41e2-a33d-5b0cf5464164 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:19.039176 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31926e41-7f0c-482c-b49d-ee97667f51e5 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:19.063000 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:19.101375 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f9611-dcd0-4e61-8c7c-cbd86fab1963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:19.102629 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f9611-dcd0-4e61-8c7c-cbd86fab1963 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:19.103103 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f9611-dcd0-4e61-8c7c-cbd86fab1963 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:19.104360 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f9611-dcd0-4e61-8c7c-cbd86fab1963 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:19.105742 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99f9611-dcd0-4e61-8c7c-cbd86fab1963 None admin] RBAC: Adding query filter params (name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:19.106502 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99f9611-dcd0-4e61-8c7c-cbd86fab1963 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:19.108334 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99f9611-dcd0-4e61-8c7c-cbd86fab1963 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:19.112430 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:19.126793 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 297/582] 10.222.0.24 () {56 vars in 1103 bytes} [Fri May 27 03:15:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:19.126793 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f9611-dcd0-4e61-8c7c-cbd86fab1963 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:19.158352 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c971f-0379-41e2-a33d-5b0cf5464164 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:19.198345 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb2dd2-32f0-472c-887d-3de886040a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:19.199278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb2dd2-32f0-472c-887d-3de886040a3b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:19.199662 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb2dd2-32f0-472c-887d-3de886040a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:19.200012 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdb2dd2-32f0-472c-887d-3de886040a3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:19.212997 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdb2dd2-32f0-472c-887d-3de886040a3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:19.215210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdb2dd2-32f0-472c-887d-3de886040a3b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:19.235081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-0cdb2dd2-32f0-472c-887d-3de886040a3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:19.235081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:19.235081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:19.235081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:19.235081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:19.235081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:19.235081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:19.235081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:19.235081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:19.235081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:19.235081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:19.235081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:19.235081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:19.235081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:19.235081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:19.236042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:19.236042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:19.236042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:19.236042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:19.236042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:19.236042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:19.236042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:19.236042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:19.236042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:19.236042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:19.236042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:19.236042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:19.236042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:19.236042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:19.236042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:19.236042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:19.236042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:19.236918 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:19.236918 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:19.236918 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:19.236918 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:19.239004 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 286/583] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:19.240215 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb2dd2-32f0-472c-887d-3de886040a3b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:19.274934 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f9611-dcd0-4e61-8c7c-cbd86fab1963 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:19.317868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b525e-8911-4ad3-aaad-446b31dc1de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:19.319560 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b525e-8911-4ad3-aaad-446b31dc1de5 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:19.319919 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b525e-8911-4ad3-aaad-446b31dc1de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:19.320268 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b525e-8911-4ad3-aaad-446b31dc1de5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:19.330381 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b525e-8911-4ad3-aaad-446b31dc1de5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:19.332592 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b525e-8911-4ad3-aaad-446b31dc1de5 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:19.348544 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdb2dd2-32f0-472c-887d-3de886040a3b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:19.350504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-698b525e-8911-4ad3-aaad-446b31dc1de5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 27 03:15:19.350504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:19.350504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:19.350504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:19.350504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:19.350504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:19.350504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:19.350504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:19.350504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:19.350504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:19.350504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:19.350504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:19.350504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:15:19.350504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:15:19.350504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:15:19.350504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:15:19.351488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:19.351488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:19.351488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:15:19.351488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:19.351488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:15:19.351488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:19.351488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:19.351488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:19.351488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:19.351488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:19.351488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:19.351488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:19.351488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:19.351488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:19.351488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:19.351488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:19.351488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:19.351488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:19.352419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:19.352419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:19.352419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:15:19.352419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:15:19.352419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:15:19.352419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:15:19.352419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:15:19.352419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:15:19.352419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 27 03:15:19.352419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:19.357119 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 298/584] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:15:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:19.357975 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b525e-8911-4ad3-aaad-446b31dc1de5 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:19.373854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:19.395439 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda79ea-cae0-47dd-9cfe-3448cb0a4b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:19.396269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda79ea-cae0-47dd-9cfe-3448cb0a4b55 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:19.396550 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda79ea-cae0-47dd-9cfe-3448cb0a4b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:19.396826 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda79ea-cae0-47dd-9cfe-3448cb0a4b55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:19.406739 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:19.410206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda79ea-cae0-47dd-9cfe-3448cb0a4b55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:19.411766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda79ea-cae0-47dd-9cfe-3448cb0a4b55 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:19.432661 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-8cda79ea-cae0-47dd-9cfe-3448cb0a4b55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:19.432661 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:19.432661 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:19.432661 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:19.432661 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:19.432661 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:19.432661 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:19.432661 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:19.432661 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:19.432661 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:19.432661 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:19.432661 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:19.432661 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:19.432661 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:19.432661 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:19.433766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:19.433766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:19.433766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:19.433766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:19.433766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:19.433766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:19.433766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:19.433766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:19.433766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:19.433766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:19.433766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:19.433766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:19.433766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:19.433766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:19.433766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:19.433766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:19.433766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:19.434658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:19.434658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:19.434658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:19.434658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:19.434658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 287/585] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:19.434658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda79ea-cae0-47dd-9cfe-3448cb0a4b55 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:19.497516 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b525e-8911-4ad3-aaad-446b31dc1de5 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:19.533252 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231719b-0bfc-44d2-a983-b44fdd862026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:19.534106 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231719b-0bfc-44d2-a983-b44fdd862026 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:19.534486 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231719b-0bfc-44d2-a983-b44fdd862026 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:19.534881 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b231719b-0bfc-44d2-a983-b44fdd862026 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:19.535734 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b231719b-0bfc-44d2-a983-b44fdd862026 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:19.536147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b231719b-0bfc-44d2-a983-b44fdd862026 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:19.537647 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b231719b-0bfc-44d2-a983-b44fdd862026 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:19.551480 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 299/586] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:19.552396 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231719b-0bfc-44d2-a983-b44fdd862026 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:19.565648 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda79ea-cae0-47dd-9cfe-3448cb0a4b55 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:19.610297 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc4b46-8ce8-473b-95a9-53c3815b6d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:19.611152 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc4b46-8ce8-473b-95a9-53c3815b6d15 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:19.611534 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc4b46-8ce8-473b-95a9-53c3815b6d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:19.611912 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc4b46-8ce8-473b-95a9-53c3815b6d15 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:19.612855 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc4b46-8ce8-473b-95a9-53c3815b6d15 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:19.613273 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc4b46-8ce8-473b-95a9-53c3815b6d15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:19.615657 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc4b46-8ce8-473b-95a9-53c3815b6d15 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:19.643712 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 288/587] 10.222.0.24 () {56 vars in 1137 bytes} [Fri May 27 03:15:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:19.644903 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc4b46-8ce8-473b-95a9-53c3815b6d15 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:19.667504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:19.675168 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b231719b-0bfc-44d2-a983-b44fdd862026 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:19.681404 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:19.717191 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159131cb-2503-4d71-bacd-9cb5d383534d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:19.718182 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159131cb-2503-4d71-bacd-9cb5d383534d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:19.718471 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159131cb-2503-4d71-bacd-9cb5d383534d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:19.718750 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159131cb-2503-4d71-bacd-9cb5d383534d None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:19.719791 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159131cb-2503-4d71-bacd-9cb5d383534d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:19.720101 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159131cb-2503-4d71-bacd-9cb5d383534d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:19.722060 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159131cb-2503-4d71-bacd-9cb5d383534d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:19.736219 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 300/588] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:19.741043 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159131cb-2503-4d71-bacd-9cb5d383534d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:19.774257 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc4b46-8ce8-473b-95a9-53c3815b6d15 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:19.820364 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d87d72-d610-41c7-aae7-70b439bf20e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:19.821621 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d87d72-d610-41c7-aae7-70b439bf20e3 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:19.822106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d87d72-d610-41c7-aae7-70b439bf20e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:19.823133 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d87d72-d610-41c7-aae7-70b439bf20e3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:19.833282 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d87d72-d610-41c7-aae7-70b439bf20e3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:19.835067 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d87d72-d610-41c7-aae7-70b439bf20e3 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:19.853614 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-c2d87d72-d610-41c7-aae7-70b439bf20e3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:15:19.853614 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:19.853614 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:19.853614 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:19.853614 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:19.853614 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:19.853614 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:19.853614 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:19.853614 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:19.853614 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:19.853614 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:19.853614 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:19.853614 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:15:19.853614 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:15:19.853614 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:15:19.853614 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:15:19.854638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:19.854638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:19.854638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:15:19.854638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:19.854638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:15:19.854638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:19.854638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:19.854638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:19.854638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:19.854638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:19.854638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:19.854638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:19.854638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:19.854638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:19.854638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:19.854638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:19.854638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:19.854638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:19.855655 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:19.855655 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:19.855655 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:15:19.855655 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:15:19.855655 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:15:19.855655 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:15:19.855655 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:15:19.855655 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:15:19.855655 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:15:19.855655 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:19.856921 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 289/589] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:15:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:19.857428 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d87d72-d610-41c7-aae7-70b439bf20e3 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:19.875332 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159131cb-2503-4d71-bacd-9cb5d383534d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:19.924721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa91c4c-126d-4e3c-b415-5b7693ad7734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:19.925793 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa91c4c-126d-4e3c-b415-5b7693ad7734 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:19.926473 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa91c4c-126d-4e3c-b415-5b7693ad7734 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:19.926757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa91c4c-126d-4e3c-b415-5b7693ad7734 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:19.942772 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa91c4c-126d-4e3c-b415-5b7693ad7734 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:19.944285 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa91c4c-126d-4e3c-b415-5b7693ad7734 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:19.956431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:19.963492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-5aa91c4c-126d-4e3c-b415-5b7693ad7734 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:19.963492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:19.963492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:19.963492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:19.963492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:19.963492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:19.963492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:19.963492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:19.963492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:19.963492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:19.963492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:19.963492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:19.963492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:19.963492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:19.963492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:19.964583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:19.964583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:19.964583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:19.964583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:19.964583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:19.964583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:19.964583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:19.964583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:19.964583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:19.964583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:19.964583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:19.964583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:19.964583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:19.964583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:19.964583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:19.964583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:19.964583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:19.965608 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:19.965608 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:19.965608 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:19.965608 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:19.969301 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 301/590] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:19.970195 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa91c4c-126d-4e3c-b415-5b7693ad7734 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:19.976288 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d87d72-d610-41c7-aae7-70b439bf20e3 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:19.980011 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:20.022597 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363c73a-ad6a-4548-86f0-75e3b00775ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:20.024082 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363c73a-ad6a-4548-86f0-75e3b00775ca None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:20.024594 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363c73a-ad6a-4548-86f0-75e3b00775ca None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:20.025816 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363c73a-ad6a-4548-86f0-75e3b00775ca None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:20.035820 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363c73a-ad6a-4548-86f0-75e3b00775ca None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:20.037448 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363c73a-ad6a-4548-86f0-75e3b00775ca None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:20.057322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-c363c73a-ad6a-4548-86f0-75e3b00775ca None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 27 03:15:20.057322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:20.057322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:20.057322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:20.057322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:20.057322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:20.057322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:20.057322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:20.057322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:20.057322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:20.057322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:20.057322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:20.057322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:15:20.057322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:15:20.057322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:15:20.057322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:15:20.058463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:20.058463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:20.058463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:15:20.058463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:20.058463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:15:20.058463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:20.058463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:20.058463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:20.058463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:20.058463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:20.058463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:20.058463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:20.058463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:20.058463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:20.058463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:20.058463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:20.058463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:20.058463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:20.059616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:20.059616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:20.059616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:15:20.059616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:15:20.059616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:15:20.059616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:15:20.059616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:15:20.059616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:15:20.059616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 27 03:15:20.059616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:20.061757 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 290/591] 10.222.0.24 () {56 vars in 1102 bytes} [Fri May 27 03:15:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:20.063032 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363c73a-ad6a-4548-86f0-75e3b00775ca None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:20.084096 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa91c4c-126d-4e3c-b415-5b7693ad7734 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:20.131998 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b5676f-1b53-4c6e-a904-466c6f679e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:20.133322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5676f-1b53-4c6e-a904-466c6f679e6f None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:20.134004 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5676f-1b53-4c6e-a904-466c6f679e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:20.134446 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5676f-1b53-4c6e-a904-466c6f679e6f None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:20.135568 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b5676f-1b53-4c6e-a904-466c6f679e6f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:20.136024 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b5676f-1b53-4c6e-a904-466c6f679e6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:20.138467 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b5676f-1b53-4c6e-a904-466c6f679e6f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:20.161916 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 302/592] 10.222.0.24 () {56 vars in 1137 bytes} [Fri May 27 03:15:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:20.163268 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b5676f-1b53-4c6e-a904-466c6f679e6f None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:20.180370 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363c73a-ad6a-4548-86f0-75e3b00775ca None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:20.221646 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cda921-655f-49e6-b88f-ed7db3597922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:20.222499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cda921-655f-49e6-b88f-ed7db3597922 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:20.222909 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cda921-655f-49e6-b88f-ed7db3597922 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:20.223290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cda921-655f-49e6-b88f-ed7db3597922 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:20.224269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cda921-655f-49e6-b88f-ed7db3597922 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:20.224682 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cda921-655f-49e6-b88f-ed7db3597922 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:20.226328 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cda921-655f-49e6-b88f-ed7db3597922 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:20.245006 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 291/593] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:20.246362 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:20.247183 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cda921-655f-49e6-b88f-ed7db3597922 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:20.257128 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:20.284510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b5676f-1b53-4c6e-a904-466c6f679e6f None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:20.322961 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4316166b-a6ff-47ab-9fe4-3366424c0a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:20.323736 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316166b-a6ff-47ab-9fe4-3366424c0a1d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:20.324018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316166b-a6ff-47ab-9fe4-3366424c0a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:20.324380 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316166b-a6ff-47ab-9fe4-3366424c0a1d None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:20.325222 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4316166b-a6ff-47ab-9fe4-3366424c0a1d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:20.325524 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4316166b-a6ff-47ab-9fe4-3366424c0a1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:20.327232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4316166b-a6ff-47ab-9fe4-3366424c0a1d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:20.345891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 303/594] 10.222.0.24 () {56 vars in 1139 bytes} [Fri May 27 03:15:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:20.346814 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4316166b-a6ff-47ab-9fe4-3366424c0a1d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:20.357370 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cda921-655f-49e6-b88f-ed7db3597922 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:20.393591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec886a1-2028-46a6-ad65-546d47f26e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:20.394490 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec886a1-2028-46a6-ad65-546d47f26e43 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:20.394782 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec886a1-2028-46a6-ad65-546d47f26e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:20.395088 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec886a1-2028-46a6-ad65-546d47f26e43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:20.406558 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec886a1-2028-46a6-ad65-546d47f26e43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:20.408182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec886a1-2028-46a6-ad65-546d47f26e43 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:20.427155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-2ec886a1-2028-46a6-ad65-546d47f26e43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:20.427155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:20.427155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:20.427155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:20.427155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:20.427155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:20.427155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:20.427155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:20.427155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:20.427155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:20.427155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:20.427155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:20.427155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:20.427155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:20.427155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:20.429232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:20.429232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:20.429232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:20.429232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:20.429232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:20.429232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:20.429232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:20.429232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:20.429232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:20.429232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:20.429232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:20.429232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:20.429232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:20.429232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:20.429232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:20.429232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:20.429232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:20.431498 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:20.431498 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:20.431498 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:20.431498 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:20.431498 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 292/595] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:20.431498 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec886a1-2028-46a6-ad65-546d47f26e43 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:20.455662 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4316166b-a6ff-47ab-9fe4-3366424c0a1d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:20.480650 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:20.492963 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:20.493375 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1281dfa-45ae-4e34-a22d-bc3210b9927f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:20.495452 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1281dfa-45ae-4e34-a22d-bc3210b9927f None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:20.495902 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1281dfa-45ae-4e34-a22d-bc3210b9927f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:20.496327 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1281dfa-45ae-4e34-a22d-bc3210b9927f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:20.506454 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1281dfa-45ae-4e34-a22d-bc3210b9927f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:20.508212 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1281dfa-45ae-4e34-a22d-bc3210b9927f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:20.527082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-a1281dfa-45ae-4e34-a22d-bc3210b9927f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:20.527082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:20.527082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:20.527082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:20.527082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:20.527082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:20.527082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:20.527082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:20.527082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:20.527082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:20.527082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:20.527082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:20.527082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:15:20.527082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:15:20.527082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:15:20.528166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:15:20.528166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:20.528166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:20.528166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:20.528166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:20.528166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:20.528166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:20.528166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:20.528166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:20.528166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:20.528166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:20.528166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:20.528166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:20.528166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:20.528166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:20.528166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:20.528166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:20.529027 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:15:20.529027 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:15:20.529027 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:15:20.529027 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:15:20.529027 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:15:20.529027 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:15:20.529027 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:20.529027 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:20.529027 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 304/596] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:20.534580 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1281dfa-45ae-4e34-a22d-bc3210b9927f None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:20.547942 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec886a1-2028-46a6-ad65-546d47f26e43 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:20.598054 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e35c0-52fc-4f4e-8ae0-2bc989e8ad4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:20.599390 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e35c0-52fc-4f4e-8ae0-2bc989e8ad4b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:20.600220 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e35c0-52fc-4f4e-8ae0-2bc989e8ad4b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:20.600639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940e35c0-52fc-4f4e-8ae0-2bc989e8ad4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:20.615833 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940e35c0-52fc-4f4e-8ae0-2bc989e8ad4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:20.617504 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940e35c0-52fc-4f4e-8ae0-2bc989e8ad4b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:20.641543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-940e35c0-52fc-4f4e-8ae0-2bc989e8ad4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:20.641543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:20.641543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:20.641543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:20.641543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:20.641543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:20.641543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:20.641543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:20.641543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:20.641543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:20.641543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:20.641543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:20.641543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:20.641543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:20.641543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:20.642639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:20.642639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:20.642639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:20.642639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:20.642639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:20.642639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:20.642639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:20.642639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:20.642639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:20.642639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:20.642639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:20.642639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:20.642639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:20.642639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:20.642639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:20.642639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:20.642639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:20.643884 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:20.643884 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:20.643884 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:20.643884 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:20.647739 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 293/597] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:20.648232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1281dfa-45ae-4e34-a22d-bc3210b9927f None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:20.648941 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e35c0-52fc-4f4e-8ae0-2bc989e8ad4b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:20.685823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ffb0bf-9eaa-4a6b-94d6-c2d25a4c5b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:20.687097 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffb0bf-9eaa-4a6b-94d6-c2d25a4c5b13 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:20.687712 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffb0bf-9eaa-4a6b-94d6-c2d25a4c5b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:20.688086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffb0bf-9eaa-4a6b-94d6-c2d25a4c5b13 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:20.689164 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ffb0bf-9eaa-4a6b-94d6-c2d25a4c5b13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:20.689576 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ffb0bf-9eaa-4a6b-94d6-c2d25a4c5b13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:20.691789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ffb0bf-9eaa-4a6b-94d6-c2d25a4c5b13 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:20.716641 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 305/598] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:20.716641 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ffb0bf-9eaa-4a6b-94d6-c2d25a4c5b13 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:20.772027 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:20.792796 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940e35c0-52fc-4f4e-8ae0-2bc989e8ad4b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:20.800433 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:20.828708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d47cd2a-d25b-41d2-88d2-fa4e6878f104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:20.829776 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47cd2a-d25b-41d2-88d2-fa4e6878f104 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:20.830216 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47cd2a-d25b-41d2-88d2-fa4e6878f104 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:20.830631 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47cd2a-d25b-41d2-88d2-fa4e6878f104 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:20.831601 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d47cd2a-d25b-41d2-88d2-fa4e6878f104 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:20.832054 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d47cd2a-d25b-41d2-88d2-fa4e6878f104 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:20.833817 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d47cd2a-d25b-41d2-88d2-fa4e6878f104 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:20.847718 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 294/599] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:15:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:20.848775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d47cd2a-d25b-41d2-88d2-fa4e6878f104 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:20.850030 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ffb0bf-9eaa-4a6b-94d6-c2d25a4c5b13 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:20.892795 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125a23a-834c-401c-833d-be12a1615779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:20.893670 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125a23a-834c-401c-833d-be12a1615779 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:20.894026 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125a23a-834c-401c-833d-be12a1615779 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:20.894745 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125a23a-834c-401c-833d-be12a1615779 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:20.895527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5125a23a-834c-401c-833d-be12a1615779 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:20.895836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5125a23a-834c-401c-833d-be12a1615779 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:20.897572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5125a23a-834c-401c-833d-be12a1615779 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:20.916777 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 306/600] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:20.916777 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125a23a-834c-401c-833d-be12a1615779 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:20.965274 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d47cd2a-d25b-41d2-88d2-fa4e6878f104 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:21.003564 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a04a8d-ebc9-44bb-8544-6024a70505c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:21.004858 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a04a8d-ebc9-44bb-8544-6024a70505c1 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:21.005353 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a04a8d-ebc9-44bb-8544-6024a70505c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:21.005872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a04a8d-ebc9-44bb-8544-6024a70505c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:21.013905 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a04a8d-ebc9-44bb-8544-6024a70505c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:21.015798 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a04a8d-ebc9-44bb-8544-6024a70505c1 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:21.024170 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125a23a-834c-401c-833d-be12a1615779 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:21.025709 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:21.031436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-72a04a8d-ebc9-44bb-8544-6024a70505c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:21.031436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:21.031436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:21.031436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:21.031436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:21.031436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:21.031436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:21.031436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:21.031436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:21.031436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:21.031436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:21.031436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:21.031436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:15:21.031436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:15:21.031436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:15:21.038413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:15:21.038413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:21.038413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:21.038413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:21.038413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:21.038413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:21.038413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:21.038413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:21.038413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:21.038413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:21.038413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:21.038413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:21.038413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:21.038413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:21.038413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:21.038413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:21.038413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:21.039344 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:15:21.039344 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:15:21.039344 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:15:21.039344 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:15:21.039344 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:15:21.039344 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:15:21.039344 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:21.039344 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:21.039344 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 295/601] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:21.039344 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a04a8d-ebc9-44bb-8544-6024a70505c1 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:21.063951 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55843a-5f89-4d1d-9f3c-a83967fe4aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:21.065152 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55843a-5f89-4d1d-9f3c-a83967fe4aff None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:21.065703 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55843a-5f89-4d1d-9f3c-a83967fe4aff None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:21.065982 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55843a-5f89-4d1d-9f3c-a83967fe4aff None admin] PATH_INFO: `/v3/projects/6ceffdd955324aec84ba048d2137ae00/users/95452d1b892a463d8e9e3eb16ca7c5ca/roles/8e315ffd561a48f0891ef53b51d548d8` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:21.066829 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:21.084119 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed55843a-5f89-4d1d-9f3c-a83967fe4aff None admin] RBAC: Authorizing `identity:create_grant(project_id=6ceffdd955324aec84ba048d2137ae00, user_id=95452d1b892a463d8e9e3eb16ca7c5ca, role_id=8e315ffd561a48f0891ef53b51d548d8)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:21.085090 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed55843a-5f89-4d1d-9f3c-a83967fe4aff None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:21.140516 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a04a8d-ebc9-44bb-8544-6024a70505c1 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:21.171557 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 307/602] 10.222.0.24 () {58 vars in 1439 bytes} [Fri May 27 03:15:20 2022] PUT /identity/v3/projects/6ceffdd955324aec84ba048d2137ae00/users/95452d1b892a463d8e9e3eb16ca7c5ca/roles/8e315ffd561a48f0891ef53b51d548d8 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:15:21.181891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755bddc-611a-4ef8-b318-aab132432eff None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:21.181891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755bddc-611a-4ef8-b318-aab132432eff None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:21.181891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755bddc-611a-4ef8-b318-aab132432eff None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:21.181891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 308/603] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:21.181891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755bddc-611a-4ef8-b318-aab132432eff None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:21.191920 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f8acc2-6d43-42da-8750-ae92bb0132a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:21.192740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8acc2-6d43-42da-8750-ae92bb0132a0 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:21.193020 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8acc2-6d43-42da-8750-ae92bb0132a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:21.193391 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8acc2-6d43-42da-8750-ae92bb0132a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:21.202474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f8acc2-6d43-42da-8750-ae92bb0132a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:21.204060 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f8acc2-6d43-42da-8750-ae92bb0132a0 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:21.220378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-c5f8acc2-6d43-42da-8750-ae92bb0132a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:21.220378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:21.220378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:21.220378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:21.220378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:21.220378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:21.220378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:21.220378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:21.220378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:21.220378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:21.220378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:21.220378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:21.220378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:15:21.220378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:15:21.220378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:15:21.221330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:15:21.221330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:21.221330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:21.221330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:21.221330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:21.221330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:21.221330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:21.221330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:21.221330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:21.221330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:21.221330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:21.221330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:21.221330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:21.221330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:21.221330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:21.221330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:21.221330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:21.222626 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:15:21.222626 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:15:21.222626 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:15:21.222626 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:15:21.222626 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:15:21.222626 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:15:21.222626 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:21.222626 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:21.222626 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 296/604] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:21.227088 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6075e-b81c-43c9-8d4c-6c57e8f62552 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:21.227503 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6075e-b81c-43c9-8d4c-6c57e8f62552 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:21.227787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6075e-b81c-43c9-8d4c-6c57e8f62552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:21.251168 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-7fb6075e-b81c-43c9-8d4c-6c57e8f62552 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:21.253019 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-7fb6075e-b81c-43c9-8d4c-6c57e8f62552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:21.308276 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755bddc-611a-4ef8-b318-aab132432eff None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:21.352319 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7fd01e-9dea-4d57-97bd-7eba0a25b42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:21.352904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7fd01e-9dea-4d57-97bd-7eba0a25b42b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:21.353179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7fd01e-9dea-4d57-97bd-7eba0a25b42b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:21.353967 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7fd01e-9dea-4d57-97bd-7eba0a25b42b None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:21.354792 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7fd01e-9dea-4d57-97bd-7eba0a25b42b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:21.355123 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7fd01e-9dea-4d57-97bd-7eba0a25b42b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:21.357154 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 297/605] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:15:21.357154 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6075e-b81c-43c9-8d4c-6c57e8f62552 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:21.360421 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7fd01e-9dea-4d57-97bd-7eba0a25b42b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:21.373986 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 309/606] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:15:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:21.375365 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c33a2-620f-4bb0-9e80-6d315fde9677 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:21.375655 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c33a2-620f-4bb0-9e80-6d315fde9677 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:21.375932 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c33a2-620f-4bb0-9e80-6d315fde9677 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:21.377390 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 310/607] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:21.382232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c33a2-620f-4bb0-9e80-6d315fde9677 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:21.386881 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:21.399293 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:21.465624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6075e-b81c-43c9-8d4c-6c57e8f62552 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:21.492542 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c33a2-620f-4bb0-9e80-6d315fde9677 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:21.500491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97d000-1208-495e-b858-c94fc66d1f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:21.501174 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97d000-1208-495e-b858-c94fc66d1f1c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:21.501454 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97d000-1208-495e-b858-c94fc66d1f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:21.501724 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97d000-1208-495e-b858-c94fc66d1f1c None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:21.502507 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e97d000-1208-495e-b858-c94fc66d1f1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:21.502813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e97d000-1208-495e-b858-c94fc66d1f1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:21.504265 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e97d000-1208-495e-b858-c94fc66d1f1c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:21.516703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 298/608] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:15:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:21.521823 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97d000-1208-495e-b858-c94fc66d1f1c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:21.528735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6e5a3f-41b0-485b-a565-73964484f060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:21.529534 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e5a3f-41b0-485b-a565-73964484f060 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:21.529819 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e5a3f-41b0-485b-a565-73964484f060 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:21.530090 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e5a3f-41b0-485b-a565-73964484f060 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:21.541197 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6e5a3f-41b0-485b-a565-73964484f060 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:21.548860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-2b6e5a3f-41b0-485b-a565-73964484f060 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:21.548860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:21.548860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:21.548860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:21.548860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:21.548860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:21.548860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:21.548860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:21.548860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:21.548860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:21.548860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:21.548860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:21.548860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:21.548860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:21.548860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:21.548860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:21.549883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:21.549883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:21.549883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:21.549883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:21.549883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:21.549883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:21.549883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:21.549883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:21.549883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:21.549883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:21.549883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:21.549883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:21.549883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:21.549883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:21.549883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:21.549883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:21.550714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:21.550714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:21.550714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:21.550714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:21.550714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:21.550714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:21.550714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:21.550714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:21.550714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:21.550714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:21.550714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:21.550714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:21.554771 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 311/609] 10.222.0.24 () {56 vars in 1102 bytes} [Fri May 27 03:15:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:21.556002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971487ca-32de-4cd4-94f6-9d7ea6d0beb9 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:21.556290 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971487ca-32de-4cd4-94f6-9d7ea6d0beb9 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:21.557107 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971487ca-32de-4cd4-94f6-9d7ea6d0beb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:21.574001 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-971487ca-32de-4cd4-94f6-9d7ea6d0beb9 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:21.575368 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-971487ca-32de-4cd4-94f6-9d7ea6d0beb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:21.610234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:21.629230 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97d000-1208-495e-b858-c94fc66d1f1c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:21.644818 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 312/610] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:15:21.645466 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971487ca-32de-4cd4-94f6-9d7ea6d0beb9 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:21.648099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:21.679128 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716bfa5-5e4c-481c-b7ec-83e9bc4dacd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:21.680289 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716bfa5-5e4c-481c-b7ec-83e9bc4dacd7 None admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:21.680571 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716bfa5-5e4c-481c-b7ec-83e9bc4dacd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:21.680984 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716bfa5-5e4c-481c-b7ec-83e9bc4dacd7 None admin] PATH_INFO: `/v3/projects/6ceffdd955324aec84ba048d2137ae00/users/c81ae2718faf4e59b799add42fb41c4f/roles/8e315ffd561a48f0891ef53b51d548d8` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:21.697880 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d716bfa5-5e4c-481c-b7ec-83e9bc4dacd7 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ceffdd955324aec84ba048d2137ae00, user_id=c81ae2718faf4e59b799add42fb41c4f, role_id=8e315ffd561a48f0891ef53b51d548d8)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:21.699360 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d716bfa5-5e4c-481c-b7ec-83e9bc4dacd7 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:21.755558 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971487ca-32de-4cd4-94f6-9d7ea6d0beb9 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:21.788696 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df500df3-a3d8-4f18-b6bc-4f8df0484a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:21.789579 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df500df3-a3d8-4f18-b6bc-4f8df0484a46 None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:21.789856 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df500df3-a3d8-4f18-b6bc-4f8df0484a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:21.790127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df500df3-a3d8-4f18-b6bc-4f8df0484a46 None admin] PATH_INFO: `/v3/projects/6ceffdd955324aec84ba048d2137ae00/users/1af0160cd3da4a0bb391b81f083715d9/roles/8e315ffd561a48f0891ef53b51d548d8` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:21.804902 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df500df3-a3d8-4f18-b6bc-4f8df0484a46 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ceffdd955324aec84ba048d2137ae00, user_id=1af0160cd3da4a0bb391b81f083715d9, role_id=8e315ffd561a48f0891ef53b51d548d8)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:21.806386 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df500df3-a3d8-4f18-b6bc-4f8df0484a46 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:21.811219 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 299/611] 10.222.0.24 () {58 vars in 1439 bytes} [Fri May 27 03:15:21 2022] PUT /identity/v3/projects/6ceffdd955324aec84ba048d2137ae00/users/c81ae2718faf4e59b799add42fb41c4f/roles/8e315ffd561a48f0891ef53b51d548d8 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:15:21.812230 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716bfa5-5e4c-481c-b7ec-83e9bc4dacd7 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:21.913851 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 313/612] 10.222.0.24 () {58 vars in 1439 bytes} [Fri May 27 03:15:21 2022] PUT /identity/v3/projects/6ceffdd955324aec84ba048d2137ae00/users/1af0160cd3da4a0bb391b81f083715d9/roles/8e315ffd561a48f0891ef53b51d548d8 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:15:21.913851 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df500df3-a3d8-4f18-b6bc-4f8df0484a46 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:21.961932 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d716bfa5-5e4c-481c-b7ec-83e9bc4dacd7 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:22.013322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:22.020307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:22.032309 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800dc841-cf97-408a-ac93-ad11fff2685a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:22.032993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800dc841-cf97-408a-ac93-ad11fff2685a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:22.033274 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800dc841-cf97-408a-ac93-ad11fff2685a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:22.033544 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800dc841-cf97-408a-ac93-ad11fff2685a None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:22.034306 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800dc841-cf97-408a-ac93-ad11fff2685a None admin] RBAC: Adding query filter params (name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:22.034613 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800dc841-cf97-408a-ac93-ad11fff2685a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:22.036114 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800dc841-cf97-408a-ac93-ad11fff2685a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:22.046823 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 300/613] 10.222.0.24 () {56 vars in 1103 bytes} [Fri May 27 03:15:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:22.052352 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800dc841-cf97-408a-ac93-ad11fff2685a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:22.113139 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df500df3-a3d8-4f18-b6bc-4f8df0484a46 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:22.179164 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6cf6cd-b9cc-429c-b955-32e859508173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:22.180518 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6cf6cd-b9cc-429c-b955-32e859508173 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:22.181983 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6cf6cd-b9cc-429c-b955-32e859508173 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:22.182424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6cf6cd-b9cc-429c-b955-32e859508173 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:22.209367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6cf6cd-b9cc-429c-b955-32e859508173 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:22.213805 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800dc841-cf97-408a-ac93-ad11fff2685a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:22.220367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-4d6cf6cd-b9cc-429c-b955-32e859508173 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:22.220367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:22.220367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:22.220367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:22.220367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:22.220367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:22.220367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:22.220367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:22.220367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:22.220367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:22.220367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:22.220367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:22.220367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:22.220367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:22.220367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:22.220367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:22.221363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:22.221363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:22.221363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:22.221363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:22.221363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:22.221363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:22.221363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:22.221363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:22.221363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:22.221363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:22.221363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:22.221363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:22.221363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:22.221363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:22.221363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:22.221363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:22.222181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:22.222181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:22.222181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:22.222181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:22.222181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:22.222181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:22.222181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:22.222181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:22.222181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:22.222181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:22.222181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:22.222181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:22.225263 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 314/614] 10.222.0.24 () {56 vars in 1102 bytes} [Fri May 27 03:15:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:22.231200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6cf6cd-b9cc-429c-b955-32e859508173 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:22.255559 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d251807-3045-44a6-8dc9-67d490c0289a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:22.256581 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d251807-3045-44a6-8dc9-67d490c0289a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:22.257088 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d251807-3045-44a6-8dc9-67d490c0289a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:22.257431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d251807-3045-44a6-8dc9-67d490c0289a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:22.275968 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d251807-3045-44a6-8dc9-67d490c0289a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:22.278232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d251807-3045-44a6-8dc9-67d490c0289a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:22.301279 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-1d251807-3045-44a6-8dc9-67d490c0289a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:22.301279 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:22.301279 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:22.301279 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:22.301279 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:22.301279 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:22.301279 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:22.301279 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:22.301279 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:22.301279 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:22.301279 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:22.301279 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:22.301279 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:22.301279 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:22.301279 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:22.302222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:22.302222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:22.302222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:22.302222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:22.302222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:22.302222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:22.302222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:22.302222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:22.302222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:22.302222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:22.302222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:22.302222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:22.302222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:22.302222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:22.302222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:22.302222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:22.302222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:22.303139 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:22.303139 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:22.303139 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:22.303139 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:22.303139 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 301/615] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:15:22.308891 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d251807-3045-44a6-8dc9-67d490c0289a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:22.360698 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6cf6cd-b9cc-429c-b955-32e859508173 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:22.376096 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:22.404344 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad135d8-c9c4-4943-8934-1fa54804d384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:22.405397 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad135d8-c9c4-4943-8934-1fa54804d384 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:22.405942 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:22.406803 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad135d8-c9c4-4943-8934-1fa54804d384 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:22.408174 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad135d8-c9c4-4943-8934-1fa54804d384 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:22.409923 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad135d8-c9c4-4943-8934-1fa54804d384 None admin] RBAC: Adding query filter params (name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:22.410614 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad135d8-c9c4-4943-8934-1fa54804d384 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:22.413510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad135d8-c9c4-4943-8934-1fa54804d384 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:22.428512 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 315/616] 10.222.0.24 () {56 vars in 1103 bytes} [Fri May 27 03:15:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:15:22.436010 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad135d8-c9c4-4943-8934-1fa54804d384 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:22.520591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d251807-3045-44a6-8dc9-67d490c0289a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:22.601478 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad135d8-c9c4-4943-8934-1fa54804d384 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:22.602587 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da5047e-d649-4975-9deb-fbfafc885589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:22.603586 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5047e-d649-4975-9deb-fbfafc885589 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:22.604079 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5047e-d649-4975-9deb-fbfafc885589 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:22.604569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5047e-d649-4975-9deb-fbfafc885589 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:22.605512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da5047e-d649-4975-9deb-fbfafc885589 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:22.606055 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da5047e-d649-4975-9deb-fbfafc885589 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:22.607720 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da5047e-d649-4975-9deb-fbfafc885589 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:22.634276 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 302/617] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:22.640318 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da5047e-d649-4975-9deb-fbfafc885589 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:22.642272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b79afc-9d9c-4989-bb53-07a3ba4bcb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:22.643201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b79afc-9d9c-4989-bb53-07a3ba4bcb5d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:22.643479 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b79afc-9d9c-4989-bb53-07a3ba4bcb5d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:22.643746 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b79afc-9d9c-4989-bb53-07a3ba4bcb5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:22.656125 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b79afc-9d9c-4989-bb53-07a3ba4bcb5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:22.662864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b79afc-9d9c-4989-bb53-07a3ba4bcb5d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:22.676523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-07b79afc-9d9c-4989-bb53-07a3ba4bcb5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:22.676523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:22.676523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:22.676523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:22.676523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:22.676523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:22.676523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:22.676523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:22.676523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:22.676523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:22.676523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:22.676523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:22.676523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:22.676523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:22.676523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:22.677451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:22.677451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:22.677451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:22.677451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:22.677451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:22.677451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:22.677451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:22.677451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:22.677451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:22.677451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:22.677451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:22.677451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:22.677451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:22.677451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:22.677451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:22.677451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:22.677451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:22.678342 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:22.678342 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:22.678342 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:22.678342 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:22.678342 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 316/618] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:22.684505 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b79afc-9d9c-4989-bb53-07a3ba4bcb5d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:22.721719 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:22.847232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da5047e-d649-4975-9deb-fbfafc885589 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:22.853438 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:22.860264 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b79afc-9d9c-4989-bb53-07a3ba4bcb5d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:22.897627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50934a7e-24f4-42e2-be15-926c5df0f4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:22.898297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50934a7e-24f4-42e2-be15-926c5df0f4e9 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:22.898616 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50934a7e-24f4-42e2-be15-926c5df0f4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:22.898914 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50934a7e-24f4-42e2-be15-926c5df0f4e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:22.899630 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50934a7e-24f4-42e2-be15-926c5df0f4e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:22.899933 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50934a7e-24f4-42e2-be15-926c5df0f4e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:22.902004 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50934a7e-24f4-42e2-be15-926c5df0f4e9 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:22.916071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 317/619] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:22.923047 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50934a7e-24f4-42e2-be15-926c5df0f4e9 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:22.937704 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbaf82-91a3-427d-bd23-86cc40064482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:22.938510 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbaf82-91a3-427d-bd23-86cc40064482 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:22.938788 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbaf82-91a3-427d-bd23-86cc40064482 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:22.939083 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbaf82-91a3-427d-bd23-86cc40064482 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:22.955613 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcbaf82-91a3-427d-bd23-86cc40064482 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:22.957094 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcbaf82-91a3-427d-bd23-86cc40064482 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:22.989119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-7dcbaf82-91a3-427d-bd23-86cc40064482 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:15:22.989119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:22.989119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:22.989119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:22.989119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:22.989119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:22.989119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:22.989119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:22.989119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:22.989119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:22.989119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:22.989119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:22.989119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:15:22.989119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:15:22.989119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:15:22.989119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:15:22.990106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:22.990106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:22.990106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:15:22.990106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:22.990106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:15:22.990106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:22.990106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:22.990106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:22.990106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:22.990106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:22.990106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:22.990106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:22.990106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:22.990106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:22.990106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:22.990106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:22.990106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:22.990106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:22.991101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:22.991101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:22.991101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:15:22.991101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:15:22.991101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:15:22.991101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:15:22.991101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:15:22.991101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:15:22.991101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:15:22.991101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:22.998966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 303/620] 10.222.0.24 () {56 vars in 1093 bytes} [Fri May 27 03:15:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:23.007928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbaf82-91a3-427d-bd23-86cc40064482 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:23.034434 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:23.050933 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50934a7e-24f4-42e2-be15-926c5df0f4e9 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:23.151816 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbaf82-91a3-427d-bd23-86cc40064482 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:23.159631 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51f9236-6d96-4c35-a69e-fe17bc6904e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:23.160440 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f9236-6d96-4c35-a69e-fe17bc6904e6 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:23.160726 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f9236-6d96-4c35-a69e-fe17bc6904e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:23.161002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51f9236-6d96-4c35-a69e-fe17bc6904e6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:23.174477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51f9236-6d96-4c35-a69e-fe17bc6904e6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:23.176917 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51f9236-6d96-4c35-a69e-fe17bc6904e6 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:23.205823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-e51f9236-6d96-4c35-a69e-fe17bc6904e6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 03:15:23.205823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:23.205823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:23.205823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:23.205823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:23.205823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:23.205823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:23.205823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:23.205823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:23.205823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:23.205823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:23.205823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:23.205823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:15:23.205823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:15:23.205823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:15:23.205823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:15:23.207017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:23.207017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:23.207017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:15:23.207017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:23.207017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:15:23.207017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:23.207017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:23.207017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:23.207017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:23.207017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:23.207017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:23.207017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:23.207017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:23.207017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:23.207017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:23.207017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:23.207017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:23.207017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:23.208147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:23.208147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:23.208147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:15:23.208147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:15:23.208147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:15:23.208147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:15:23.208147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:15:23.208147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:15:23.208147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 03:15:23.208147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:23.214929 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7581d-ff32-4b4d-acb9-2ea96cc46752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:23.215774 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7581d-ff32-4b4d-acb9-2ea96cc46752 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:23.216739 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 318/621] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:15:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:23.217169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7581d-ff32-4b4d-acb9-2ea96cc46752 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:23.217635 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7581d-ff32-4b4d-acb9-2ea96cc46752 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:23.218700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b7581d-ff32-4b4d-acb9-2ea96cc46752 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:23.219198 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b7581d-ff32-4b4d-acb9-2ea96cc46752 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:23.220776 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b7581d-ff32-4b4d-acb9-2ea96cc46752 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:23.222817 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51f9236-6d96-4c35-a69e-fe17bc6904e6 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:23.255847 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 304/622] 10.222.0.24 () {56 vars in 1133 bytes} [Fri May 27 03:15:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:15:23.262737 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7581d-ff32-4b4d-acb9-2ea96cc46752 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:23.301336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:23.397664 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51f9236-6d96-4c35-a69e-fe17bc6904e6 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:23.435026 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db64373-8f61-4a71-83bd-e4240bcfe64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:23.436019 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db64373-8f61-4a71-83bd-e4240bcfe64c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:23.436440 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db64373-8f61-4a71-83bd-e4240bcfe64c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:23.436898 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db64373-8f61-4a71-83bd-e4240bcfe64c None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:23.437999 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db64373-8f61-4a71-83bd-e4240bcfe64c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:23.438453 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db64373-8f61-4a71-83bd-e4240bcfe64c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:23.440057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db64373-8f61-4a71-83bd-e4240bcfe64c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:23.461849 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 319/623] 10.222.0.24 () {56 vars in 1135 bytes} [Fri May 27 03:15:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:15:23.468068 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db64373-8f61-4a71-83bd-e4240bcfe64c None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:23.472783 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:23.491444 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7581d-ff32-4b4d-acb9-2ea96cc46752 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:23.534415 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8fa632-6031-4d22-b68b-04fcc9978a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:23.536433 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fa632-6031-4d22-b68b-04fcc9978a9e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:23.536877 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fa632-6031-4d22-b68b-04fcc9978a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:23.537572 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fa632-6031-4d22-b68b-04fcc9978a9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:23.641370 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8fa632-6031-4d22-b68b-04fcc9978a9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:23.642867 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8fa632-6031-4d22-b68b-04fcc9978a9e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:23.681012 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:23.683194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db64373-8f61-4a71-83bd-e4240bcfe64c None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:23.747236 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b080d-f56c-47b8-93b9-df5682da19e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:23.748033 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b080d-f56c-47b8-93b9-df5682da19e5 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:23.748309 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b080d-f56c-47b8-93b9-df5682da19e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:23.748577 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b080d-f56c-47b8-93b9-df5682da19e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:23.769608 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757b080d-f56c-47b8-93b9-df5682da19e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:23.771257 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757b080d-f56c-47b8-93b9-df5682da19e5 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:23.790253 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-757b080d-f56c-47b8-93b9-df5682da19e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:23.790253 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:23.790253 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:23.790253 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:23.790253 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:23.790253 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:23.790253 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:23.790253 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:23.790253 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:23.790253 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:23.790253 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:23.790253 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:23.790253 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:23.790253 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:23.790253 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:23.791340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:23.791340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:23.791340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:23.791340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:23.791340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:23.791340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:23.791340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:23.791340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:23.791340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:23.791340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:23.791340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:23.791340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:23.791340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:23.791340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:23.791340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:23.791340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:23.791340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:23.792393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:23.792393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:23.792393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:23.792393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:23.797051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 305/625] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:15:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:15:23.798010 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 320/625] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:23.798393 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b4ebd-4f10-44b8-a9b2-d578a8004eff None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:23.798683 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b4ebd-4f10-44b8-a9b2-d578a8004eff None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:23.798984 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b4ebd-4f10-44b8-a9b2-d578a8004eff None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:23.799795 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b080d-f56c-47b8-93b9-df5682da19e5 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:23.800472 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 306/626] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:23.805495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b4ebd-4f10-44b8-a9b2-d578a8004eff None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:23.869578 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:23.914299 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b080d-f56c-47b8-93b9-df5682da19e5 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:23.930578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b4ebd-4f10-44b8-a9b2-d578a8004eff None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:23.957713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:23.964876 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d09fca-2c23-4e6a-a3ce-d3a0ff1f8fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:23.965665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d09fca-2c23-4e6a-a3ce-d3a0ff1f8fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:23.965952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d09fca-2c23-4e6a-a3ce-d3a0ff1f8fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:23.966236 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d09fca-2c23-4e6a-a3ce-d3a0ff1f8fd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:23.966992 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d09fca-2c23-4e6a-a3ce-d3a0ff1f8fd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:23.967302 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d09fca-2c23-4e6a-a3ce-d3a0ff1f8fd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:23.968724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d09fca-2c23-4e6a-a3ce-d3a0ff1f8fd9 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:23.969562 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda484fd-7b14-4531-b19b-2de78a6095c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:23.970564 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda484fd-7b14-4531-b19b-2de78a6095c9 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:23.971029 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda484fd-7b14-4531-b19b-2de78a6095c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:23.971378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda484fd-7b14-4531-b19b-2de78a6095c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:23.982248 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda484fd-7b14-4531-b19b-2de78a6095c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:23.982392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 321/627] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:23.983671 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56970c82-17ff-4820-a3a6-803a74d9fb1b None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:23.983957 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56970c82-17ff-4820-a3a6-803a74d9fb1b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:23.984232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56970c82-17ff-4820-a3a6-803a74d9fb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:23.988286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda484fd-7b14-4531-b19b-2de78a6095c9 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:24.003845 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-56970c82-17ff-4820-a3a6-803a74d9fb1b None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:24.005081 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-56970c82-17ff-4820-a3a6-803a74d9fb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:24.011325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-bda484fd-7b14-4531-b19b-2de78a6095c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:24.011325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:24.011325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:24.011325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:24.011325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:24.011325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:24.011325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:24.011325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:24.011325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:24.011325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:24.011325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:24.011325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:24.011325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:24.011325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:24.011325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:24.012882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:24.012882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:24.012882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:24.012882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:24.012882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:24.012882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:24.012882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:24.012882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:24.012882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:24.012882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:24.012882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:24.012882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:24.012882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:24.012882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:24.012882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:24.012882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:24.012882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:24.014724 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:24.014724 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:24.014724 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:24.014724 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:24.015950 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 307/628] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:24.017252 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda484fd-7b14-4531-b19b-2de78a6095c9 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:24.098856 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 322/629] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:15:24.098856 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56970c82-17ff-4820-a3a6-803a74d9fb1b None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:24.147263 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:24.216249 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56970c82-17ff-4820-a3a6-803a74d9fb1b None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:24.274514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91780ca6-51d9-4091-b711-e60bfc4b5c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:24.275785 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91780ca6-51d9-4091-b711-e60bfc4b5c2c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:24.276070 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91780ca6-51d9-4091-b711-e60bfc4b5c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:24.276343 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91780ca6-51d9-4091-b711-e60bfc4b5c2c None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:24.277090 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91780ca6-51d9-4091-b711-e60bfc4b5c2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:24.277397 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91780ca6-51d9-4091-b711-e60bfc4b5c2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:24.278779 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91780ca6-51d9-4091-b711-e60bfc4b5c2c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:24.294304 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 323/630] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:24.295183 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91780ca6-51d9-4091-b711-e60bfc4b5c2c None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:24.300976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda484fd-7b14-4531-b19b-2de78a6095c9 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:24.340151 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d583c1e-79e5-4b13-99fb-d590b42640b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:24.341232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d583c1e-79e5-4b13-99fb-d590b42640b3 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:24.341695 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d583c1e-79e5-4b13-99fb-d590b42640b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:24.342168 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d583c1e-79e5-4b13-99fb-d590b42640b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:24.349289 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d583c1e-79e5-4b13-99fb-d590b42640b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:24.350995 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d583c1e-79e5-4b13-99fb-d590b42640b3 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:24.365585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-0d583c1e-79e5-4b13-99fb-d590b42640b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:24.365585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:24.365585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:24.365585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:24.365585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:24.365585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:24.365585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:24.365585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:24.365585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:24.365585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:24.365585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:24.365585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:24.365585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:15:24.365585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:15:24.365585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:15:24.366548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:15:24.366548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:24.366548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:24.366548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:24.366548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:24.366548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:24.366548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:24.366548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:24.366548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:24.366548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:24.366548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:24.366548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:24.366548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:24.366548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:24.366548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:24.366548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:24.366548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:24.367648 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:15:24.367648 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:15:24.367648 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:15:24.367648 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:15:24.367648 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:15:24.367648 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:15:24.367648 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:24.367648 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:24.370024 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 308/631] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:24.371582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdd7ec-3756-4934-a0a3-2d3a82752846 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:24.372067 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdd7ec-3756-4934-a0a3-2d3a82752846 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:24.372526 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdd7ec-3756-4934-a0a3-2d3a82752846 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:24.378210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 309/632] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:24.379305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcdd7ec-3756-4934-a0a3-2d3a82752846 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:24.421258 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:24.423146 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91780ca6-51d9-4091-b711-e60bfc4b5c2c None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:24.431900 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:24.462311 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7853466-1496-4b57-8bbd-271cbc6325e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:24.463244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7853466-1496-4b57-8bbd-271cbc6325e4 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:24.463532 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7853466-1496-4b57-8bbd-271cbc6325e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:24.463813 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7853466-1496-4b57-8bbd-271cbc6325e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:24.476667 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7853466-1496-4b57-8bbd-271cbc6325e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:24.484505 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-b7853466-1496-4b57-8bbd-271cbc6325e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:24.484505 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:24.484505 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:24.484505 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:24.484505 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:24.484505 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:24.484505 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:24.484505 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:24.484505 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:24.484505 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:24.484505 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:24.484505 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:24.484505 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:24.484505 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:24.484505 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:24.484505 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:24.485449 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:24.485449 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:24.485449 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:24.485449 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:24.485449 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:24.485449 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:24.485449 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:24.485449 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:24.485449 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:24.485449 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:24.485449 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:24.485449 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:24.485449 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:24.485449 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:24.485449 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:24.485449 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:24.486296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:24.486296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:24.486296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:24.486296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:24.486296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:24.486296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:24.486296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:24.486296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:24.486296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:24.486296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:24.486296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:24.486296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:24.486296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 324/633] 10.222.0.24 () {56 vars in 1102 bytes} [Fri May 27 03:15:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:24.487036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7853466-1496-4b57-8bbd-271cbc6325e4 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:24.515377 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcdd7ec-3756-4934-a0a3-2d3a82752846 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:24.554120 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a08fc-75d0-420d-8194-67ebfd7711f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:24.555892 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a08fc-75d0-420d-8194-67ebfd7711f4 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:24.556183 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a08fc-75d0-420d-8194-67ebfd7711f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:24.556972 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a08fc-75d0-420d-8194-67ebfd7711f4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:24.564988 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a08fc-75d0-420d-8194-67ebfd7711f4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:24.566436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a08fc-75d0-420d-8194-67ebfd7711f4 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:24.584906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-f77a08fc-75d0-420d-8194-67ebfd7711f4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:24.584906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:24.584906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:24.584906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:24.584906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:24.584906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:24.584906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:24.584906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:24.584906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:24.584906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:24.584906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:24.584906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:24.584906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:15:24.584906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:15:24.584906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:15:24.585854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:15:24.585854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:24.585854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:24.585854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:24.585854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:24.585854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:24.585854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:24.585854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:24.585854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:24.585854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:24.585854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:24.585854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:24.585854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:24.585854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:24.585854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:24.585854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:24.585854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:24.586740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:15:24.586740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:15:24.586740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:15:24.586740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:15:24.586740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:15:24.586740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:15:24.586740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:24.586740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:24.586740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 310/634] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:24.587756 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff726eb-147c-4538-ad2b-5c31e6381025 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:24.587756 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff726eb-147c-4538-ad2b-5c31e6381025 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:24.588025 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff726eb-147c-4538-ad2b-5c31e6381025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:24.610103 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-1ff726eb-147c-4538-ad2b-5c31e6381025 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:24.611245 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1ff726eb-147c-4538-ad2b-5c31e6381025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:24.614968 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7853466-1496-4b57-8bbd-271cbc6325e4 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:24.650796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbfa76-1086-4c55-a142-849aab90c082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:24.651501 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbfa76-1086-4c55-a142-849aab90c082 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:24.651786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbfa76-1086-4c55-a142-849aab90c082 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:24.652061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbfa76-1086-4c55-a142-849aab90c082 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:24.652846 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bbfa76-1086-4c55-a142-849aab90c082 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:24.653155 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bbfa76-1086-4c55-a142-849aab90c082 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:24.654584 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bbfa76-1086-4c55-a142-849aab90c082 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:24.665981 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 325/635] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:15:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:24.666757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbfa76-1086-4c55-a142-849aab90c082 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:24.684416 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 311/636] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:15:24.685581 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0b3d4-1c66-41d1-a4fd-c9ba81b0e418 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:24.685866 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0b3d4-1c66-41d1-a4fd-c9ba81b0e418 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:24.686136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0b3d4-1c66-41d1-a4fd-c9ba81b0e418 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:24.687417 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 312/637] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:24.688107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a0b3d4-1c66-41d1-a4fd-c9ba81b0e418 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:24.705883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:24.717344 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:24.776369 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbfa76-1086-4c55-a142-849aab90c082 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:24.792920 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a0b3d4-1c66-41d1-a4fd-c9ba81b0e418 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:24.809427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773615f4-4a27-42f5-968b-53256fca311a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:24.810091 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773615f4-4a27-42f5-968b-53256fca311a None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:24.810374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773615f4-4a27-42f5-968b-53256fca311a None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:24.810650 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773615f4-4a27-42f5-968b-53256fca311a None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:24.811424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773615f4-4a27-42f5-968b-53256fca311a None admin] RBAC: Adding query filter params (name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:24.811733 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773615f4-4a27-42f5-968b-53256fca311a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:24.813089 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773615f4-4a27-42f5-968b-53256fca311a None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:24.820828 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 326/638] 10.222.0.24 () {56 vars in 1103 bytes} [Fri May 27 03:15:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:24.821735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773615f4-4a27-42f5-968b-53256fca311a None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:24.826343 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98151df9-bcb7-4090-a3f0-c1739938d572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:24.827035 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98151df9-bcb7-4090-a3f0-c1739938d572 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:24.827333 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98151df9-bcb7-4090-a3f0-c1739938d572 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:24.827581 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98151df9-bcb7-4090-a3f0-c1739938d572 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:24.828366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98151df9-bcb7-4090-a3f0-c1739938d572 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:24.828676 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98151df9-bcb7-4090-a3f0-c1739938d572 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:24.830475 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98151df9-bcb7-4090-a3f0-c1739938d572 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:24.842830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 313/639] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:15:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:24.844073 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4e9ee-7304-4a4c-a974-4c2bb40bb79c None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:24.844357 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4e9ee-7304-4a4c-a974-4c2bb40bb79c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:24.844628 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4e9ee-7304-4a4c-a974-4c2bb40bb79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:24.862231 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-60a4e9ee-7304-4a4c-a974-4c2bb40bb79c None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:24.863364 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-60a4e9ee-7304-4a4c-a974-4c2bb40bb79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:24.910896 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:24.923227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773615f4-4a27-42f5-968b-53256fca311a None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:24.927430 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 314/640] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:15:24.941907 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4e9ee-7304-4a4c-a974-4c2bb40bb79c None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:24.955660 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:24.958723 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e7c2f-1894-4835-aa71-8c2cba3c6df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:24.959580 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e7c2f-1894-4835-aa71-8c2cba3c6df1 None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:24.959862 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e7c2f-1894-4835-aa71-8c2cba3c6df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:24.960137 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e7c2f-1894-4835-aa71-8c2cba3c6df1 None admin] PATH_INFO: `/v3/projects/6ceffdd955324aec84ba048d2137ae00/users/21d559cd7127490da84185f20684929d/roles/8e315ffd561a48f0891ef53b51d548d8` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:24.975988 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51e7c2f-1894-4835-aa71-8c2cba3c6df1 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ceffdd955324aec84ba048d2137ae00, user_id=21d559cd7127490da84185f20684929d, role_id=8e315ffd561a48f0891ef53b51d548d8)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:24.977442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51e7c2f-1894-4835-aa71-8c2cba3c6df1 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:25.037585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a4e9ee-7304-4a4c-a974-4c2bb40bb79c None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:25.045171 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 327/641] 10.222.0.24 () {58 vars in 1439 bytes} [Fri May 27 03:15:24 2022] PUT /identity/v3/projects/6ceffdd955324aec84ba048d2137ae00/users/21d559cd7127490da84185f20684929d/roles/8e315ffd561a48f0891ef53b51d548d8 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:15:25.045969 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e7c2f-1894-4835-aa71-8c2cba3c6df1 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:25.072720 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a3de9-28fa-49cd-9601-482b0e423a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:25.073899 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a3de9-28fa-49cd-9601-482b0e423a85 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:25.074367 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a3de9-28fa-49cd-9601-482b0e423a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:25.074815 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a3de9-28fa-49cd-9601-482b0e423a85 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:25.091023 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a3de9-28fa-49cd-9601-482b0e423a85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:25.099568 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-da1a3de9-28fa-49cd-9601-482b0e423a85 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:25.099568 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:25.099568 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:25.099568 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:25.099568 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:25.099568 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:25.099568 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:25.099568 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:25.099568 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:25.099568 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:25.099568 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:25.099568 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:25.099568 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:25.099568 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:25.099568 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:25.099568 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:25.100792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:25.100792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:25.100792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:25.100792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:25.100792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:25.100792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:25.100792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:25.100792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:25.100792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:25.100792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:25.100792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:25.100792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:25.100792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:25.100792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:25.100792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:25.100792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:25.101679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:25.101679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:25.101679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:25.101679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:25.101679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:25.101679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:25.101679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:25.101679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:25.101679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:25.101679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:25.101679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:25.101679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:25.101679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 315/642] 10.222.0.24 () {56 vars in 1102 bytes} [Fri May 27 03:15:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:25.102372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a3de9-28fa-49cd-9601-482b0e423a85 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:25.187973 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:25.217448 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a3de9-28fa-49cd-9601-482b0e423a85 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:25.258274 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f02585-adb2-4dd1-8f06-9b8a7b01ed9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:25.258991 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f02585-adb2-4dd1-8f06-9b8a7b01ed9d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:25.259274 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f02585-adb2-4dd1-8f06-9b8a7b01ed9d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:25.260129 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f02585-adb2-4dd1-8f06-9b8a7b01ed9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:25.260964 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f02585-adb2-4dd1-8f06-9b8a7b01ed9d None admin] RBAC: Adding query filter params (role.id=8e315ffd561a48f0891ef53b51d548d8, scope.project.id=6ceffdd955324aec84ba048d2137ae00, user.id=22a9592143014f3287cd75c60ca17237) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:25.261278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f02585-adb2-4dd1-8f06-9b8a7b01ed9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:25.262708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f02585-adb2-4dd1-8f06-9b8a7b01ed9d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:25.268931 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 316/643] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:15:25 2022] GET /identity/v3/role_assignments?user.id=22a9592143014f3287cd75c60ca17237&scope.project.id=6ceffdd955324aec84ba048d2137ae00&role.id=8e315ffd561a48f0891ef53b51d548d8 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:25.269861 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f02585-adb2-4dd1-8f06-9b8a7b01ed9d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:25.328235 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e7c2f-1894-4835-aa71-8c2cba3c6df1 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:25.367671 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725dfe43-f6d9-404e-a37d-7cd5671786cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:25.368485 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725dfe43-f6d9-404e-a37d-7cd5671786cf None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:25.368792 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725dfe43-f6d9-404e-a37d-7cd5671786cf None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:25.369063 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725dfe43-f6d9-404e-a37d-7cd5671786cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:25.376767 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:25.381393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725dfe43-f6d9-404e-a37d-7cd5671786cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:25.382901 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725dfe43-f6d9-404e-a37d-7cd5671786cf None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:25.389846 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f02585-adb2-4dd1-8f06-9b8a7b01ed9d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:25.403324 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-725dfe43-f6d9-404e-a37d-7cd5671786cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:25.403324 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:25.403324 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:25.403324 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:25.403324 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:25.403324 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:25.403324 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:25.403324 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:25.403324 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:25.403324 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:25.403324 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:25.403324 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:25.403324 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:25.403324 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:25.403324 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:25.404345 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:25.404345 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:25.404345 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:25.404345 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:25.404345 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:25.404345 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:25.404345 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:25.404345 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:25.404345 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:25.404345 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:25.404345 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:25.404345 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:25.404345 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:25.404345 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:25.404345 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:25.404345 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:25.404345 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:25.405250 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:25.405250 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:25.405250 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:25.405250 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:25.405250 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 328/644] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:25.405988 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725dfe43-f6d9-404e-a37d-7cd5671786cf None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:25.437615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932997c-fb4c-441c-b2b0-a75ce73e26f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:25.439095 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932997c-fb4c-441c-b2b0-a75ce73e26f4 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:25.439958 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932997c-fb4c-441c-b2b0-a75ce73e26f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:25.440471 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932997c-fb4c-441c-b2b0-a75ce73e26f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:25.451715 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:25.457558 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1932997c-fb4c-441c-b2b0-a75ce73e26f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:25.466247 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-1932997c-fb4c-441c-b2b0-a75ce73e26f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:25.466247 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:25.466247 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:25.466247 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:25.466247 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:25.466247 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:25.466247 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:25.466247 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:25.466247 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:25.466247 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:25.466247 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:25.466247 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:25.466247 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:25.466247 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:25.466247 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:25.466247 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:25.467225 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:25.467225 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:25.467225 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:25.467225 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:25.467225 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:25.467225 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:25.467225 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:25.467225 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:25.467225 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:25.467225 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:25.467225 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:25.467225 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:25.467225 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:25.467225 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:25.467225 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:25.467225 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:25.468188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:25.468188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:25.468188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:25.468188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:25.468188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:25.468188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:25.468188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:25.468188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:25.468188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:25.468188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:25.468188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:25.468188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:25.470237 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 317/645] 10.222.0.24 () {56 vars in 1102 bytes} [Fri May 27 03:15:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:25.472119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932997c-fb4c-441c-b2b0-a75ce73e26f4 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:25.530124 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725dfe43-f6d9-404e-a37d-7cd5671786cf None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:25.570783 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf9dbf-81cc-4e0f-99da-80f42777eefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:25.571615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf9dbf-81cc-4e0f-99da-80f42777eefc None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:25.571954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf9dbf-81cc-4e0f-99da-80f42777eefc None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:25.572227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf9dbf-81cc-4e0f-99da-80f42777eefc None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:25.572985 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf9dbf-81cc-4e0f-99da-80f42777eefc None admin] RBAC: Adding query filter params (name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:25.573292 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf9dbf-81cc-4e0f-99da-80f42777eefc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:25.574631 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf9dbf-81cc-4e0f-99da-80f42777eefc None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:25.585930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 329/646] 10.222.0.24 () {56 vars in 1103 bytes} [Fri May 27 03:15:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:25.586787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf9dbf-81cc-4e0f-99da-80f42777eefc None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:25.592632 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932997c-fb4c-441c-b2b0-a75ce73e26f4 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:25.636347 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcb98e-cf93-4317-b52d-1d42435c8a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:25.637077 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcb98e-cf93-4317-b52d-1d42435c8a64 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:25.637456 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcb98e-cf93-4317-b52d-1d42435c8a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:25.637734 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bcb98e-cf93-4317-b52d-1d42435c8a64 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:25.638566 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bcb98e-cf93-4317-b52d-1d42435c8a64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:25.638894 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bcb98e-cf93-4317-b52d-1d42435c8a64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:25.640268 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bcb98e-cf93-4317-b52d-1d42435c8a64 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:25.644418 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:25.653676 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 318/647] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:25.654716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcb98e-cf93-4317-b52d-1d42435c8a64 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:25.698102 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf9dbf-81cc-4e0f-99da-80f42777eefc None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:25.729693 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:25.733159 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5e513-747d-4d2d-abf8-0be0e4c0d10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:25.733920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5e513-747d-4d2d-abf8-0be0e4c0d10c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:25.734269 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5e513-747d-4d2d-abf8-0be0e4c0d10c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:25.734609 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5e513-747d-4d2d-abf8-0be0e4c0d10c None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:25.735459 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5e513-747d-4d2d-abf8-0be0e4c0d10c None admin] RBAC: Adding query filter params (name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:25.735830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5e513-747d-4d2d-abf8-0be0e4c0d10c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:25.737343 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5e513-747d-4d2d-abf8-0be0e4c0d10c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:25.746217 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 330/648] 10.222.0.24 () {56 vars in 1103 bytes} [Fri May 27 03:15:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:25.747182 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5e513-747d-4d2d-abf8-0be0e4c0d10c None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:25.766104 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bcb98e-cf93-4317-b52d-1d42435c8a64 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:25.803264 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80d999-d05c-4c84-8619-09aab16095ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:25.804182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80d999-d05c-4c84-8619-09aab16095ae None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:25.804465 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80d999-d05c-4c84-8619-09aab16095ae None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:25.804738 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd80d999-d05c-4c84-8619-09aab16095ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:25.816006 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd80d999-d05c-4c84-8619-09aab16095ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:25.817517 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd80d999-d05c-4c84-8619-09aab16095ae None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:25.837716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-bd80d999-d05c-4c84-8619-09aab16095ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:25.837716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:25.837716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:25.837716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:25.837716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:25.837716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:25.837716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:25.837716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:25.837716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:25.837716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:25.837716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:25.837716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:25.837716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:25.837716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:25.837716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:25.838696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:25.838696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:25.838696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:25.838696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:25.838696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:25.838696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:25.838696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:25.838696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:25.838696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:25.838696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:25.838696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:25.838696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:25.838696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:25.838696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:25.838696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:25.838696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:25.838696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:25.840089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:25.840089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:25.840089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:25.840089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:25.840089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 319/649] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:25.840089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80d999-d05c-4c84-8619-09aab16095ae None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:25.850591 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5e513-747d-4d2d-abf8-0be0e4c0d10c None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:25.865656 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:25.886727 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1252b3-ee75-472d-a377-39ffcce08f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:25.887752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1252b3-ee75-472d-a377-39ffcce08f84 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:25.888114 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1252b3-ee75-472d-a377-39ffcce08f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:25.888484 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1252b3-ee75-472d-a377-39ffcce08f84 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:25.896528 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1252b3-ee75-472d-a377-39ffcce08f84 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:25.898414 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1252b3-ee75-472d-a377-39ffcce08f84 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:25.913859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-8d1252b3-ee75-472d-a377-39ffcce08f84 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 27 03:15:25.913859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:25.913859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:25.913859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:25.913859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:25.913859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:25.913859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:25.913859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:25.913859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:25.913859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:25.913859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:25.913859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:25.913859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:15:25.913859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:15:25.913859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:15:25.913859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:15:25.914887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:25.914887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:25.914887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:15:25.914887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:25.914887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:15:25.914887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:25.914887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:25.914887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:25.914887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:25.914887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:25.914887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:25.914887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:25.914887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:25.914887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:25.914887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:25.914887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:25.914887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:25.914887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:25.916404 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:25.916404 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:25.916404 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:15:25.916404 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:15:25.916404 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:15:25.916404 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:15:25.916404 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:15:25.916404 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:15:25.916404 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 27 03:15:25.916404 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:25.920711 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 331/650] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:15:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:25.921891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1252b3-ee75-472d-a377-39ffcce08f84 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:25.953791 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd80d999-d05c-4c84-8619-09aab16095ae None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:25.979308 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:25.991063 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b058e4-086e-4fe6-a142-17a7da754584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:25.992070 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b058e4-086e-4fe6-a142-17a7da754584 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:25.992547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b058e4-086e-4fe6-a142-17a7da754584 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:25.992989 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b058e4-086e-4fe6-a142-17a7da754584 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:26.005198 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b058e4-086e-4fe6-a142-17a7da754584 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:26.007030 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b058e4-086e-4fe6-a142-17a7da754584 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:26.026152 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1252b3-ee75-472d-a377-39ffcce08f84 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:26.026447 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-11b058e4-086e-4fe6-a142-17a7da754584 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:26.026447 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:26.026447 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:26.026447 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:26.026447 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:26.026447 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:26.026447 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:26.026447 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:26.026447 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:26.026447 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:26.026447 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:26.026447 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:26.026447 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:26.026447 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:26.026447 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:26.027305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:26.027305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:26.027305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:26.027305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:26.027305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:26.027305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:26.027305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:26.027305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:26.027305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:26.027305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:26.027305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:26.027305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:26.027305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:26.027305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:26.027305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:26.027305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:26.027305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:26.028190 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:26.028190 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:26.028190 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:26.028190 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:26.030076 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 320/651] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:26.031078 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b058e4-086e-4fe6-a142-17a7da754584 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:26.060890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df1641-51f4-49f7-8a63-bb8c32cf3691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:26.061642 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df1641-51f4-49f7-8a63-bb8c32cf3691 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:26.061992 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df1641-51f4-49f7-8a63-bb8c32cf3691 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:26.062333 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df1641-51f4-49f7-8a63-bb8c32cf3691 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:26.063179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df1641-51f4-49f7-8a63-bb8c32cf3691 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:26.063549 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df1641-51f4-49f7-8a63-bb8c32cf3691 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:26.064984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7df1641-51f4-49f7-8a63-bb8c32cf3691 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:26.085920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 332/652] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:26.086871 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df1641-51f4-49f7-8a63-bb8c32cf3691 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:26.121198 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:26.142665 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b058e4-086e-4fe6-a142-17a7da754584 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:26.179111 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292a137-5af6-4ef3-8f0a-a9c7769bb015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:26.180216 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292a137-5af6-4ef3-8f0a-a9c7769bb015 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:26.180744 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292a137-5af6-4ef3-8f0a-a9c7769bb015 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:26.181855 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292a137-5af6-4ef3-8f0a-a9c7769bb015 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:26.183012 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c292a137-5af6-4ef3-8f0a-a9c7769bb015 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:26.183521 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c292a137-5af6-4ef3-8f0a-a9c7769bb015 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:26.185324 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c292a137-5af6-4ef3-8f0a-a9c7769bb015 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:26.187499 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df1641-51f4-49f7-8a63-bb8c32cf3691 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:26.205016 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 321/653] 10.222.0.24 () {56 vars in 1137 bytes} [Fri May 27 03:15:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:26.206204 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292a137-5af6-4ef3-8f0a-a9c7769bb015 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:26.227585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120f9f6-b419-46a5-aec1-cfd9df9ad6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:26.228325 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120f9f6-b419-46a5-aec1-cfd9df9ad6c1 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:26.228605 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120f9f6-b419-46a5-aec1-cfd9df9ad6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:26.228874 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120f9f6-b419-46a5-aec1-cfd9df9ad6c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:26.229587 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6120f9f6-b419-46a5-aec1-cfd9df9ad6c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:26.229964 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6120f9f6-b419-46a5-aec1-cfd9df9ad6c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:26.231582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6120f9f6-b419-46a5-aec1-cfd9df9ad6c1 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:26.239201 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:26.250122 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 333/654] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:26.251292 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120f9f6-b419-46a5-aec1-cfd9df9ad6c1 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:26.340090 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292a137-5af6-4ef3-8f0a-a9c7769bb015 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:26.361139 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:26.368075 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120f9f6-b419-46a5-aec1-cfd9df9ad6c1 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:26.390062 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7f676-3aed-48f0-ba84-c98894a6d34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:26.391156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7f676-3aed-48f0-ba84-c98894a6d34c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:26.391433 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7f676-3aed-48f0-ba84-c98894a6d34c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:26.391862 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7f676-3aed-48f0-ba84-c98894a6d34c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:26.402466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7f676-3aed-48f0-ba84-c98894a6d34c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:26.403912 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cbbb99-e0b6-41ba-8b06-499494aed9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:26.404054 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7f676-3aed-48f0-ba84-c98894a6d34c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:26.404824 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbbb99-e0b6-41ba-8b06-499494aed9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:26.405110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbbb99-e0b6-41ba-8b06-499494aed9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:26.405384 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbbb99-e0b6-41ba-8b06-499494aed9e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:26.422713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-12d7f676-3aed-48f0-ba84-c98894a6d34c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:15:26.422713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:26.422713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:26.422713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:26.422713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:26.422713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:26.422713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:26.422713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:26.422713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:26.422713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:26.422713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:26.422713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:26.422713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:15:26.422713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:15:26.422713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:15:26.422713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:15:26.423668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:26.423668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:26.423668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:15:26.423668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:26.423668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:15:26.423668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:26.423668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:26.423668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:26.423668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:26.423668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:26.423668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:26.423668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:26.423668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:26.423668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:26.423668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:26.423668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:26.423668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:26.423668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:26.424583 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:26.424583 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:26.424583 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:15:26.424583 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:15:26.424583 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:15:26.424583 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:15:26.424583 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:15:26.424583 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:15:26.424583 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:15:26.424583 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:26.424583 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 322/655] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:15:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:26.425765 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7f676-3aed-48f0-ba84-c98894a6d34c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:26.475977 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cbbb99-e0b6-41ba-8b06-499494aed9e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:26.477349 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cbbb99-e0b6-41ba-8b06-499494aed9e1 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:26.528479 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:26.549826 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7f676-3aed-48f0-ba84-c98894a6d34c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:26.592541 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2b966-656a-4a11-8866-baeeb833a294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:26.593336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2b966-656a-4a11-8866-baeeb833a294 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:26.593935 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2b966-656a-4a11-8866-baeeb833a294 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:26.593935 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2b966-656a-4a11-8866-baeeb833a294 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:26.594583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:26.602462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 334/656] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:15:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:15:26.602682 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa2b966-656a-4a11-8866-baeeb833a294 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:26.603940 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cbbb99-e0b6-41ba-8b06-499494aed9e1 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:26.608798 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa2b966-656a-4a11-8866-baeeb833a294 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:26.624169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-1fa2b966-656a-4a11-8866-baeeb833a294 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 27 03:15:26.624169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:26.624169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:26.624169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:26.624169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:26.624169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:26.624169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:26.624169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:26.624169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:26.624169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:26.624169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:26.624169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:26.624169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:15:26.624169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:15:26.624169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:15:26.624169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:15:26.626449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:26.626449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:26.626449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:15:26.626449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:26.626449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:15:26.626449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:26.626449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:26.626449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:26.626449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:26.626449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:26.626449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:26.626449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:26.626449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:26.626449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:26.626449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:26.626449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:26.626449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:26.626449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:26.628850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:26.628850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:26.628850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:15:26.628850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:15:26.628850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:15:26.628850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:15:26.628850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:15:26.628850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:15:26.628850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 27 03:15:26.628850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:26.628850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 323/657] 10.222.0.24 () {56 vars in 1102 bytes} [Fri May 27 03:15:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:26.628850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2b966-656a-4a11-8866-baeeb833a294 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:26.721607 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cbbb99-e0b6-41ba-8b06-499494aed9e1 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:26.745007 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2b966-656a-4a11-8866-baeeb833a294 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:26.767449 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b56b8-fc9c-4d38-aa82-41618efe85bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:26.768368 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b56b8-fc9c-4d38-aa82-41618efe85bb None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:26.769082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b56b8-fc9c-4d38-aa82-41618efe85bb None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:26.769374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b56b8-fc9c-4d38-aa82-41618efe85bb None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:26.770492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b56b8-fc9c-4d38-aa82-41618efe85bb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:26.770811 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b56b8-fc9c-4d38-aa82-41618efe85bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:26.773170 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b56b8-fc9c-4d38-aa82-41618efe85bb None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:26.780901 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d769dc8-b7e0-42e3-8cbf-b2d0bf1692fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:26.781697 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d769dc8-b7e0-42e3-8cbf-b2d0bf1692fe None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:26.781980 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d769dc8-b7e0-42e3-8cbf-b2d0bf1692fe None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:26.782256 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d769dc8-b7e0-42e3-8cbf-b2d0bf1692fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:26.791274 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:26.795600 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d769dc8-b7e0-42e3-8cbf-b2d0bf1692fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:26.797355 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d769dc8-b7e0-42e3-8cbf-b2d0bf1692fe None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:26.800727 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 335/658] 10.222.0.24 () {56 vars in 1137 bytes} [Fri May 27 03:15:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:26.806789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b56b8-fc9c-4d38-aa82-41618efe85bb None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:26.822850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-3d769dc8-b7e0-42e3-8cbf-b2d0bf1692fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:26.822850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:26.822850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:26.822850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:26.822850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:26.822850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:26.822850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:26.822850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:26.822850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:26.822850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:26.822850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:26.822850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:26.822850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:26.822850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:26.822850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:26.823766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:26.823766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:26.823766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:26.823766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:26.823766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:26.823766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:26.823766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:26.823766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:26.823766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:26.823766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:26.823766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:26.823766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:26.823766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:26.823766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:26.823766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:26.823766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:26.823766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:26.824646 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:26.824646 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:26.824646 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:26.824646 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:26.827716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 324/659] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:26.829414 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d769dc8-b7e0-42e3-8cbf-b2d0bf1692fe None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:26.870889 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:26.920676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b56b8-fc9c-4d38-aa82-41618efe85bb None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:26.936882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d769dc8-b7e0-42e3-8cbf-b2d0bf1692fe None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:26.958231 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625bb3ea-bad9-4002-b578-1bf864e90df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:26.959282 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625bb3ea-bad9-4002-b578-1bf864e90df4 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:26.959750 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625bb3ea-bad9-4002-b578-1bf864e90df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:26.960280 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625bb3ea-bad9-4002-b578-1bf864e90df4 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:26.961286 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625bb3ea-bad9-4002-b578-1bf864e90df4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:26.961773 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625bb3ea-bad9-4002-b578-1bf864e90df4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:26.963625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625bb3ea-bad9-4002-b578-1bf864e90df4 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:26.973849 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c7dd2-07a0-4c56-b6f5-c5091db44309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:26.977614 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c7dd2-07a0-4c56-b6f5-c5091db44309 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:26.977614 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c7dd2-07a0-4c56-b6f5-c5091db44309 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:26.977614 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c7dd2-07a0-4c56-b6f5-c5091db44309 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:26.985732 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 336/660] 10.222.0.24 () {56 vars in 1139 bytes} [Fri May 27 03:15:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:26.987004 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625bb3ea-bad9-4002-b578-1bf864e90df4 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:27.044444 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:27.054354 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c7dd2-07a0-4c56-b6f5-c5091db44309 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:27.056077 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c7dd2-07a0-4c56-b6f5-c5091db44309 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:27.102257 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625bb3ea-bad9-4002-b578-1bf864e90df4 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:27.114962 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:27.141130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec55397-ccde-4a0d-8683-c5101506c8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:27.142129 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec55397-ccde-4a0d-8683-c5101506c8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:27.142690 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec55397-ccde-4a0d-8683-c5101506c8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:27.143161 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec55397-ccde-4a0d-8683-c5101506c8c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:27.144152 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec55397-ccde-4a0d-8683-c5101506c8c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:27.145526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec55397-ccde-4a0d-8683-c5101506c8c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:27.147219 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec55397-ccde-4a0d-8683-c5101506c8c2 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:27.160242 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 337/661] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:27.161387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec55397-ccde-4a0d-8683-c5101506c8c2 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:27.164427 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 325/662] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:15:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:15:27.165374 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c7dd2-07a0-4c56-b6f5-c5091db44309 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:27.264181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:27.273056 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c7dd2-07a0-4c56-b6f5-c5091db44309 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:27.277192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec55397-ccde-4a0d-8683-c5101506c8c2 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:27.312931 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9071a49-56f7-4b3a-a74b-a66ec5bf58b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:27.313745 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9071a49-56f7-4b3a-a74b-a66ec5bf58b0 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:27.314095 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9071a49-56f7-4b3a-a74b-a66ec5bf58b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:27.314501 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9071a49-56f7-4b3a-a74b-a66ec5bf58b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:27.319025 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf00cf-062a-4461-870d-86385b6a1219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:27.319879 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf00cf-062a-4461-870d-86385b6a1219 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:27.320358 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf00cf-062a-4461-870d-86385b6a1219 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:27.320820 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf00cf-062a-4461-870d-86385b6a1219 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:27.322345 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9071a49-56f7-4b3a-a74b-a66ec5bf58b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:27.323913 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9071a49-56f7-4b3a-a74b-a66ec5bf58b0 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:27.339234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-e9071a49-56f7-4b3a-a74b-a66ec5bf58b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:27.339234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:27.339234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:27.339234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:27.339234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:27.339234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:27.339234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:27.339234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:27.339234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:27.339234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:27.339234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:27.339234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:27.339234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:15:27.339234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:15:27.339234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:15:27.340181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:15:27.340181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:27.340181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:27.340181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:27.340181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:27.340181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:27.340181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:27.340181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:27.340181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:27.340181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:27.340181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:27.340181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:27.340181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:27.340181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:27.340181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:27.340181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:27.340181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:27.341478 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:15:27.341478 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:15:27.341478 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:15:27.341478 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:15:27.341478 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:15:27.341478 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:15:27.341478 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:27.341478 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:27.341478 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 326/663] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:27.341944 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9071a49-56f7-4b3a-a74b-a66ec5bf58b0 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:27.369412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:27.401111 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf00cf-062a-4461-870d-86385b6a1219 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:27.403220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf00cf-062a-4461-870d-86385b6a1219 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:27.446250 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9071a49-56f7-4b3a-a74b-a66ec5bf58b0 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:27.483338 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384068d-8ae3-4d9f-b9d9-0cfad81b0eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:27.484234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384068d-8ae3-4d9f-b9d9-0cfad81b0eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:27.484512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384068d-8ae3-4d9f-b9d9-0cfad81b0eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:27.484783 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384068d-8ae3-4d9f-b9d9-0cfad81b0eb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:27.497046 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a384068d-8ae3-4d9f-b9d9-0cfad81b0eb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:27.498994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a384068d-8ae3-4d9f-b9d9-0cfad81b0eb5 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:27.509125 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 338/664] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:15:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:15:27.510804 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf00cf-062a-4461-870d-86385b6a1219 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:27.518110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-a384068d-8ae3-4d9f-b9d9-0cfad81b0eb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:27.518110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:27.518110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:27.518110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:27.518110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:27.518110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:27.518110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:27.518110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:27.518110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:27.518110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:27.518110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:27.518110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:27.518110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:27.518110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:27.518110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:27.519119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:27.519119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:27.519119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:27.519119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:27.519119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:27.519119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:27.519119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:27.519119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:27.519119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:27.519119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:27.519119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:27.519119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:27.519119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:27.519119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:27.519119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:27.519119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:27.519119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:27.520023 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:27.520023 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:27.520023 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:27.520023 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:27.523207 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 327/665] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:27.526248 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d33a5-d029-499b-a59e-6ac6c2718f45 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:27.526616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d33a5-d029-499b-a59e-6ac6c2718f45 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:27.526979 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d33a5-d029-499b-a59e-6ac6c2718f45 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:27.528561 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 328/666] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:27.533478 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d33a5-d029-499b-a59e-6ac6c2718f45 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:27.538777 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:27.589763 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:27.625352 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf00cf-062a-4461-870d-86385b6a1219 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:27.651986 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d33a5-d029-499b-a59e-6ac6c2718f45 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:27.668061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5766fe-7ef0-4d6f-ad8e-c85568fcfd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:27.668748 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5766fe-7ef0-4d6f-ad8e-c85568fcfd86 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:27.669032 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5766fe-7ef0-4d6f-ad8e-c85568fcfd86 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:27.669306 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5766fe-7ef0-4d6f-ad8e-c85568fcfd86 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:27.670092 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5766fe-7ef0-4d6f-ad8e-c85568fcfd86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:27.670403 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5766fe-7ef0-4d6f-ad8e-c85568fcfd86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:27.671948 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5766fe-7ef0-4d6f-ad8e-c85568fcfd86 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:27.688529 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 339/667] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:15:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:27.688529 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5766fe-7ef0-4d6f-ad8e-c85568fcfd86 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:27.693021 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ca26f2-98fb-400d-b1c6-168bce79178c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:27.693819 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca26f2-98fb-400d-b1c6-168bce79178c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:27.694103 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca26f2-98fb-400d-b1c6-168bce79178c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:27.694377 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ca26f2-98fb-400d-b1c6-168bce79178c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:27.705294 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ca26f2-98fb-400d-b1c6-168bce79178c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:27.706738 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ca26f2-98fb-400d-b1c6-168bce79178c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:27.725172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-60ca26f2-98fb-400d-b1c6-168bce79178c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:27.725172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:27.725172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:27.725172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:27.725172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:27.725172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:27.725172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:27.725172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:27.725172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:27.725172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:27.725172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:27.725172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:27.725172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:27.725172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:27.725172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:27.726273 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:27.726273 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:27.726273 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:27.726273 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:27.726273 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:27.726273 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:27.726273 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:27.726273 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:27.726273 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:27.726273 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:27.726273 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:27.726273 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:27.726273 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:27.726273 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:27.726273 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:27.726273 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:27.726273 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:27.727227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:27.727227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:27.727227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:27.727227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:27.727227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 329/668] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:27.727973 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55807fc7-895f-4726-9c22-197f1e9155e3 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:27.728826 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55807fc7-895f-4726-9c22-197f1e9155e3 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:27.729106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55807fc7-895f-4726-9c22-197f1e9155e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:27.745249 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-55807fc7-895f-4726-9c22-197f1e9155e3 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:27.746434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-55807fc7-895f-4726-9c22-197f1e9155e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:27.790155 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5766fe-7ef0-4d6f-ad8e-c85568fcfd86 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:27.794743 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:27.810201 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 330/669] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:15:27.811043 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55807fc7-895f-4726-9c22-197f1e9155e3 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:27.818548 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:27.824598 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938d4503-9770-458f-817c-2549d8cd0f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:27.825064 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d4503-9770-458f-817c-2549d8cd0f50 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:27.825348 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d4503-9770-458f-817c-2549d8cd0f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:27.825954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938d4503-9770-458f-817c-2549d8cd0f50 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:27.826599 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938d4503-9770-458f-817c-2549d8cd0f50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:27.826933 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938d4503-9770-458f-817c-2549d8cd0f50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:27.828347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938d4503-9770-458f-817c-2549d8cd0f50 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:27.844312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 340/670] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:27.844312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195729c0-724f-4abb-9634-ffff21ff5570 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:27.844312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195729c0-724f-4abb-9634-ffff21ff5570 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:27.844312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195729c0-724f-4abb-9634-ffff21ff5570 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:27.844312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 341/671] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:27.844312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195729c0-724f-4abb-9634-ffff21ff5570 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:27.913815 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55807fc7-895f-4726-9c22-197f1e9155e3 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:27.950051 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195729c0-724f-4abb-9634-ffff21ff5570 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:27.955924 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc35e4-7e34-4530-8bf7-fbac4889d536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:27.956609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc35e4-7e34-4530-8bf7-fbac4889d536 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:27.956888 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc35e4-7e34-4530-8bf7-fbac4889d536 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:27.957168 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc35e4-7e34-4530-8bf7-fbac4889d536 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:27.957992 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc35e4-7e34-4530-8bf7-fbac4889d536 None admin] RBAC: Adding query filter params (role.id=8e315ffd561a48f0891ef53b51d548d8, scope.project.id=6ceffdd955324aec84ba048d2137ae00, user.id=95452d1b892a463d8e9e3eb16ca7c5ca) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:27.958297 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc35e4-7e34-4530-8bf7-fbac4889d536 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:27.959751 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc35e4-7e34-4530-8bf7-fbac4889d536 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:27.965647 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 331/672] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:15:27 2022] GET /identity/v3/role_assignments?user.id=95452d1b892a463d8e9e3eb16ca7c5ca&scope.project.id=6ceffdd955324aec84ba048d2137ae00&role.id=8e315ffd561a48f0891ef53b51d548d8 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:27.966559 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc35e4-7e34-4530-8bf7-fbac4889d536 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:27.987705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215d8b8-9843-4f8a-b1c1-ba1ed96947da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:27.988172 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215d8b8-9843-4f8a-b1c1-ba1ed96947da None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:27.988456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215d8b8-9843-4f8a-b1c1-ba1ed96947da None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:27.988734 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215d8b8-9843-4f8a-b1c1-ba1ed96947da None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:27.989476 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215d8b8-9843-4f8a-b1c1-ba1ed96947da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:27.989784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215d8b8-9843-4f8a-b1c1-ba1ed96947da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:27.991559 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215d8b8-9843-4f8a-b1c1-ba1ed96947da None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:28.005438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 342/673] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:28.008853 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215d8b8-9843-4f8a-b1c1-ba1ed96947da None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:28.023965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:28.066788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:28.092221 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc35e4-7e34-4530-8bf7-fbac4889d536 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:28.125748 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215d8b8-9843-4f8a-b1c1-ba1ed96947da None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:28.131380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f8f9d5-855a-4ecf-a456-28b8e7b5ac5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:28.132227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8f9d5-855a-4ecf-a456-28b8e7b5ac5a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:28.132509 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8f9d5-855a-4ecf-a456-28b8e7b5ac5a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:28.132778 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8f9d5-855a-4ecf-a456-28b8e7b5ac5a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:28.145281 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f8f9d5-855a-4ecf-a456-28b8e7b5ac5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:28.153237 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-45f8f9d5-855a-4ecf-a456-28b8e7b5ac5a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:28.153237 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:28.153237 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:28.153237 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:28.153237 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:28.153237 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:28.153237 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:28.153237 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:28.153237 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:28.153237 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:28.153237 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:28.153237 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:28.153237 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:28.153237 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:28.153237 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:28.153237 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:28.154314 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:28.154314 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:28.154314 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:28.154314 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:28.154314 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:28.154314 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:28.154314 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:28.154314 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:28.154314 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:28.154314 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:28.154314 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:28.154314 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:28.154314 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:28.154314 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:28.154314 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:28.154314 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:28.160182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:28.160182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:28.160182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:28.160182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:28.160182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:28.160182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:28.160182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:28.160182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:28.160182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:28.160182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:28.160182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:28.160182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:28.160182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 332/674] 10.222.0.24 () {56 vars in 1102 bytes} [Fri May 27 03:15:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:28.160182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939904c7-302c-4117-895a-e59d43927bca None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:28.161085 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939904c7-302c-4117-895a-e59d43927bca None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:28.161085 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939904c7-302c-4117-895a-e59d43927bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:28.173700 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162131ad-963b-43da-a8a3-308a7766372c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:28.174498 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162131ad-963b-43da-a8a3-308a7766372c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:28.174775 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162131ad-963b-43da-a8a3-308a7766372c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:28.175140 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162131ad-963b-43da-a8a3-308a7766372c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:28.176042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-939904c7-302c-4117-895a-e59d43927bca None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:28.177245 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-939904c7-302c-4117-895a-e59d43927bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:28.182123 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162131ad-963b-43da-a8a3-308a7766372c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:28.183606 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162131ad-963b-43da-a8a3-308a7766372c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:28.199312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-162131ad-963b-43da-a8a3-308a7766372c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:28.199312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:28.199312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:28.199312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:28.199312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:28.199312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:28.199312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:28.199312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:28.199312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:28.199312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:28.199312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:28.199312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:28.199312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:15:28.199312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:15:28.199312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:15:28.200385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:15:28.200385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:28.200385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:28.200385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:28.200385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:28.200385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:28.200385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:28.200385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:28.200385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:28.200385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:28.200385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:28.200385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:28.200385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:28.200385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:28.200385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:28.200385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:28.200385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:28.201722 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:15:28.201722 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:15:28.201722 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:15:28.201722 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:15:28.201722 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:15:28.201722 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:15:28.201722 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:28.201722 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:28.201722 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 343/675] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:28.202314 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162131ad-963b-43da-a8a3-308a7766372c None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:28.252865 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 333/676] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:15:28.253785 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939904c7-302c-4117-895a-e59d43927bca None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:28.302275 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:28.334215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162131ad-963b-43da-a8a3-308a7766372c None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:28.347359 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:28.363674 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939904c7-302c-4117-895a-e59d43927bca None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:28.370976 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38484be8-6b44-4b49-8fbe-cb1417d8083c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:28.371679 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38484be8-6b44-4b49-8fbe-cb1417d8083c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:28.371969 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38484be8-6b44-4b49-8fbe-cb1417d8083c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:28.372235 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38484be8-6b44-4b49-8fbe-cb1417d8083c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:28.379907 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38484be8-6b44-4b49-8fbe-cb1417d8083c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:28.381394 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38484be8-6b44-4b49-8fbe-cb1417d8083c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:28.395256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-38484be8-6b44-4b49-8fbe-cb1417d8083c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:28.395256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:28.395256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:28.395256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:28.395256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:28.395256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:28.395256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:28.395256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:28.395256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:28.395256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:28.395256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:28.395256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:28.395256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:15:28.395256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:15:28.395256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:15:28.396131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:15:28.396131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:28.396131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:28.396131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:28.396131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:28.396131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:28.396131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:28.396131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:28.396131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:28.396131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:28.396131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:28.396131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:28.396131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:28.396131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:28.396131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:28.396131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:28.396131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:28.397012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:15:28.397012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:15:28.397012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:15:28.397012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:15:28.397012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:15:28.397012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:15:28.397012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:28.397012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:28.397012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 344/677] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:28.397500 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38484be8-6b44-4b49-8fbe-cb1417d8083c None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:28.401099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bc9eb-bf3b-4951-b87b-f5848965e7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:28.401775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bc9eb-bf3b-4951-b87b-f5848965e7ea None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:28.402057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bc9eb-bf3b-4951-b87b-f5848965e7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:28.402330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bc9eb-bf3b-4951-b87b-f5848965e7ea None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:28.403117 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941bc9eb-bf3b-4951-b87b-f5848965e7ea None admin] RBAC: Adding query filter params (name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:28.403423 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941bc9eb-bf3b-4951-b87b-f5848965e7ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:28.404798 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941bc9eb-bf3b-4951-b87b-f5848965e7ea None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:28.413367 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 334/678] 10.222.0.24 () {56 vars in 1103 bytes} [Fri May 27 03:15:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:28.414162 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bc9eb-bf3b-4951-b87b-f5848965e7ea None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:28.515495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bc9eb-bf3b-4951-b87b-f5848965e7ea None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:28.516172 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38484be8-6b44-4b49-8fbe-cb1417d8083c None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:28.517132 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:28.550991 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9939761-8a41-403e-a168-c0f337425ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:28.552085 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9939761-8a41-403e-a168-c0f337425ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:28.552363 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9939761-8a41-403e-a168-c0f337425ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:28.552636 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9939761-8a41-403e-a168-c0f337425ec8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:28.557049 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9c4d1-81b9-4a17-961b-af71a643b3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:28.557700 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9c4d1-81b9-4a17-961b-af71a643b3bc None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:28.557977 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9c4d1-81b9-4a17-961b-af71a643b3bc None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:28.558247 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9c4d1-81b9-4a17-961b-af71a643b3bc None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:28.559054 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b9c4d1-81b9-4a17-961b-af71a643b3bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:28.559360 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b9c4d1-81b9-4a17-961b-af71a643b3bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:28.560774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b9c4d1-81b9-4a17-961b-af71a643b3bc None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:28.564037 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9939761-8a41-403e-a168-c0f337425ec8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:28.571804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-b9939761-8a41-403e-a168-c0f337425ec8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:15:28.571804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:28.571804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:28.571804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:28.571804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:28.571804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:28.571804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:28.571804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:28.571804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:28.571804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:28.571804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:28.571804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:28.571804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:28.571804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:28.571804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:28.571804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:28.572717 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:28.572717 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:28.572717 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:28.572717 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:28.572717 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:28.572717 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:28.572717 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:28.572717 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:28.572717 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:28.572717 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:28.572717 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:28.572717 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:28.572717 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:28.572717 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:28.572717 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:28.572717 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:28.573559 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:28.573559 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:28.573559 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:28.573559 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:28.573559 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:28.573559 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:28.573559 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:28.573559 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:28.573559 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:28.573559 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:28.573559 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:15:28.573559 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:28.573559 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 335/679] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:15:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:28.574243 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9939761-8a41-403e-a168-c0f337425ec8 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:28.578111 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 345/680] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:15:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:28.578908 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9c4d1-81b9-4a17-961b-af71a643b3bc None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:28.610054 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:28.683241 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9c4d1-81b9-4a17-961b-af71a643b3bc None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:28.683840 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9939761-8a41-403e-a168-c0f337425ec8 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:28.718849 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07ac0f-ee8f-4f2b-bb2f-cadf04505c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:28.719657 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07ac0f-ee8f-4f2b-bb2f-cadf04505c10 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:28.719935 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07ac0f-ee8f-4f2b-bb2f-cadf04505c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:28.720204 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07ac0f-ee8f-4f2b-bb2f-cadf04505c10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:28.721708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a09a45-b885-4f80-a700-15570cafdc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:28.722373 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a09a45-b885-4f80-a700-15570cafdc3b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:28.722655 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a09a45-b885-4f80-a700-15570cafdc3b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:28.723187 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a09a45-b885-4f80-a700-15570cafdc3b None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:28.723801 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a09a45-b885-4f80-a700-15570cafdc3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:28.724120 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a09a45-b885-4f80-a700-15570cafdc3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:28.725562 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a09a45-b885-4f80-a700-15570cafdc3b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:28.730884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07ac0f-ee8f-4f2b-bb2f-cadf04505c10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:28.732314 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07ac0f-ee8f-4f2b-bb2f-cadf04505c10 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:28.742707 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 336/681] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:15:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:28.742707 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a09a45-b885-4f80-a700-15570cafdc3b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:28.754200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-ef07ac0f-ee8f-4f2b-bb2f-cadf04505c10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:28.754200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:28.754200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:28.754200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:28.754200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:28.754200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:28.754200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:28.754200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:28.754200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:28.754200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:28.754200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:28.754200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:28.754200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:28.754200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:28.754200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:28.755424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:28.755424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:28.755424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:28.755424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:28.755424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:28.755424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:28.755424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:28.755424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:28.755424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:28.755424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:28.755424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:28.755424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:28.755424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:28.755424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:28.755424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:28.755424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:28.755424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:28.756290 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:28.756290 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:28.756290 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:28.756290 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:28.760416 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 346/682] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:28.761156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07ac0f-ee8f-4f2b-bb2f-cadf04505c10 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:28.768221 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:28.846395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a09a45-b885-4f80-a700-15570cafdc3b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:28.854486 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:28.873852 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07ac0f-ee8f-4f2b-bb2f-cadf04505c10 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:28.883831 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446653e-3f23-4fe6-8274-540bb89efdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:28.883831 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446653e-3f23-4fe6-8274-540bb89efdf9 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:28.883831 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446653e-3f23-4fe6-8274-540bb89efdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:28.884680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446653e-3f23-4fe6-8274-540bb89efdf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:28.885443 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0446653e-3f23-4fe6-8274-540bb89efdf9 None admin] RBAC: Adding query filter params (role.id=8e315ffd561a48f0891ef53b51d548d8, scope.project.id=6ceffdd955324aec84ba048d2137ae00, user.id=c81ae2718faf4e59b799add42fb41c4f) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:28.885753 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0446653e-3f23-4fe6-8274-540bb89efdf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:28.887196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0446653e-3f23-4fe6-8274-540bb89efdf9 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:28.893187 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 337/683] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:15:28 2022] GET /identity/v3/role_assignments?user.id=c81ae2718faf4e59b799add42fb41c4f&scope.project.id=6ceffdd955324aec84ba048d2137ae00&role.id=8e315ffd561a48f0891ef53b51d548d8 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:28.900193 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446653e-3f23-4fe6-8274-540bb89efdf9 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:28.914161 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b62d7-a973-47f0-bb6c-1ce1a176b830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:28.915021 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b62d7-a973-47f0-bb6c-1ce1a176b830 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:28.915907 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b62d7-a973-47f0-bb6c-1ce1a176b830 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:28.916360 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b62d7-a973-47f0-bb6c-1ce1a176b830 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:28.917623 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b62d7-a973-47f0-bb6c-1ce1a176b830 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:28.917754 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b62d7-a973-47f0-bb6c-1ce1a176b830 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:28.919747 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b62d7-a973-47f0-bb6c-1ce1a176b830 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:28.934318 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 347/684] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:15:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:28.934318 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b62d7-a973-47f0-bb6c-1ce1a176b830 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:29.016502 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:29.036433 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446653e-3f23-4fe6-8274-540bb89efdf9 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:29.076259 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b62d7-a973-47f0-bb6c-1ce1a176b830 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:29.076579 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df686f9-fc23-4b1e-9f81-6b54468470af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:29.077352 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df686f9-fc23-4b1e-9f81-6b54468470af None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:29.077734 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df686f9-fc23-4b1e-9f81-6b54468470af None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:29.078500 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df686f9-fc23-4b1e-9f81-6b54468470af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:29.080436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df686f9-fc23-4b1e-9f81-6b54468470af None admin] RBAC: Adding query filter params (role.id=8e315ffd561a48f0891ef53b51d548d8, scope.project.id=6ceffdd955324aec84ba048d2137ae00, user.id=1af0160cd3da4a0bb391b81f083715d9) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:29.080564 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df686f9-fc23-4b1e-9f81-6b54468470af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:29.084500 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df686f9-fc23-4b1e-9f81-6b54468470af None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:29.092450 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 338/685] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:15:28 2022] GET /identity/v3/role_assignments?user.id=1af0160cd3da4a0bb391b81f083715d9&scope.project.id=6ceffdd955324aec84ba048d2137ae00&role.id=8e315ffd561a48f0891ef53b51d548d8 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:29.093382 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df686f9-fc23-4b1e-9f81-6b54468470af None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:29.124386 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb765d0-fbf0-4773-bb9f-a80c24bea71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:29.124386 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb765d0-fbf0-4773-bb9f-a80c24bea71b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:29.124691 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb765d0-fbf0-4773-bb9f-a80c24bea71b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:29.124904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb765d0-fbf0-4773-bb9f-a80c24bea71b None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:29.125863 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb765d0-fbf0-4773-bb9f-a80c24bea71b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:29.126036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb765d0-fbf0-4773-bb9f-a80c24bea71b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:29.127538 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb765d0-fbf0-4773-bb9f-a80c24bea71b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:29.142004 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 348/686] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:29.147527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb765d0-fbf0-4773-bb9f-a80c24bea71b None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:29.152280 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:29.222228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df686f9-fc23-4b1e-9f81-6b54468470af None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:29.262494 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a281a3-daa1-4112-b596-90cf76865723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:29.263619 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb765d0-fbf0-4773-bb9f-a80c24bea71b None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:29.263900 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a281a3-daa1-4112-b596-90cf76865723 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:29.264353 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a281a3-daa1-4112-b596-90cf76865723 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:29.264806 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a281a3-daa1-4112-b596-90cf76865723 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:29.279595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a281a3-daa1-4112-b596-90cf76865723 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:29.281494 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a281a3-daa1-4112-b596-90cf76865723 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:29.301838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-09a281a3-daa1-4112-b596-90cf76865723 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:29.301838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:29.301838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:29.301838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:29.301838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:29.301838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:29.301838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:29.301838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:29.301838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:29.301838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:29.301838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:29.301838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:29.301838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:29.301838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:29.301838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:29.302787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:29.302787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:29.302787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:29.302787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:29.302787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:29.302787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:29.302787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:29.302787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:29.302787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:29.302787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:29.302787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:29.302787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:29.302787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:29.302787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:29.302787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:29.302787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:29.302787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:29.303709 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:29.303709 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:29.303709 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:29.303709 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:29.306864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 339/687] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:29.307697 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777aa8d0-7eb0-4d76-8ab5-2da4c3406b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:29.308500 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777aa8d0-7eb0-4d76-8ab5-2da4c3406b17 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:29.308776 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777aa8d0-7eb0-4d76-8ab5-2da4c3406b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:29.309048 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777aa8d0-7eb0-4d76-8ab5-2da4c3406b17 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:29.311177 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a281a3-daa1-4112-b596-90cf76865723 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:29.315068 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:29.323756 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777aa8d0-7eb0-4d76-8ab5-2da4c3406b17 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:29.325671 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777aa8d0-7eb0-4d76-8ab5-2da4c3406b17 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:29.346469 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-777aa8d0-7eb0-4d76-8ab5-2da4c3406b17 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 03:15:29.346469 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:29.346469 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:29.346469 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:29.346469 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:29.346469 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:29.346469 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:29.346469 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:29.346469 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:29.346469 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:29.346469 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:29.346469 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:29.346469 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:15:29.346469 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:15:29.346469 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:15:29.346469 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:15:29.347693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:29.347693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:29.347693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:15:29.347693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:29.347693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:15:29.347693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:29.347693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:29.347693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:29.347693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:29.347693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:29.347693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:29.347693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:29.347693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:29.347693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:29.347693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:29.347693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:29.347693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:29.347693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:29.348836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:29.348836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:29.348836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:15:29.348836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:15:29.348836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:15:29.348836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:15:29.348836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:15:29.348836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:15:29.348836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 03:15:29.348836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:29.351687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 349/688] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:15:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:29.365792 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777aa8d0-7eb0-4d76-8ab5-2da4c3406b17 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:29.438967 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a281a3-daa1-4112-b596-90cf76865723 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:29.440676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:29.475744 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edeb618-a4c4-4a02-b9f0-91682944d301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:29.476635 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edeb618-a4c4-4a02-b9f0-91682944d301 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:29.477066 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edeb618-a4c4-4a02-b9f0-91682944d301 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:29.477498 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edeb618-a4c4-4a02-b9f0-91682944d301 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:29.478923 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edeb618-a4c4-4a02-b9f0-91682944d301 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:29.479370 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edeb618-a4c4-4a02-b9f0-91682944d301 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:29.480881 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edeb618-a4c4-4a02-b9f0-91682944d301 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:29.482779 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777aa8d0-7eb0-4d76-8ab5-2da4c3406b17 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:29.494517 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 340/689] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:15:29.501903 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edeb618-a4c4-4a02-b9f0-91682944d301 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:29.521619 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523ab92-35f5-4626-9eac-c9c9d64b1fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:29.522300 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523ab92-35f5-4626-9eac-c9c9d64b1fdd None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:29.522581 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523ab92-35f5-4626-9eac-c9c9d64b1fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:29.522914 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523ab92-35f5-4626-9eac-c9c9d64b1fdd None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:29.524395 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e523ab92-35f5-4626-9eac-c9c9d64b1fdd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:29.525034 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e523ab92-35f5-4626-9eac-c9c9d64b1fdd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:29.527257 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e523ab92-35f5-4626-9eac-c9c9d64b1fdd None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:29.564764 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 350/690] 10.222.0.24 () {56 vars in 1135 bytes} [Fri May 27 03:15:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:29.569871 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523ab92-35f5-4626-9eac-c9c9d64b1fdd None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:29.585153 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:29.624775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edeb618-a4c4-4a02-b9f0-91682944d301 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:29.660186 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfdffdd-7b8a-4039-812c-557fc6c8525d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:29.660981 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfdffdd-7b8a-4039-812c-557fc6c8525d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:29.661261 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfdffdd-7b8a-4039-812c-557fc6c8525d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:29.661536 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfdffdd-7b8a-4039-812c-557fc6c8525d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:29.671005 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfdffdd-7b8a-4039-812c-557fc6c8525d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:29.672466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfdffdd-7b8a-4039-812c-557fc6c8525d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:29.687195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-fbfdffdd-7b8a-4039-812c-557fc6c8525d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:15:29.687195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:29.687195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:29.687195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:29.687195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:29.687195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:29.687195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:29.687195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:29.687195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:29.687195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:29.687195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:29.687195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:29.687195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:15:29.687195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:15:29.687195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:15:29.687195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:15:29.688591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:29.688591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:29.688591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:15:29.688591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:29.688591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:15:29.688591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:29.688591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:29.688591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:29.688591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:29.688591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:29.688591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:29.688591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:29.688591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:29.688591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:29.688591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:29.688591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:29.688591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:29.688591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:29.689616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:29.689616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:29.689616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:15:29.689616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:15:29.689616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:15:29.689616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:15:29.689616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:15:29.689616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:15:29.689616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:15:29.689616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:29.689616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 341/691] 10.222.0.24 () {56 vars in 1093 bytes} [Fri May 27 03:15:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:29.694085 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfdffdd-7b8a-4039-812c-557fc6c8525d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:29.781876 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523ab92-35f5-4626-9eac-c9c9d64b1fdd None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:29.797413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfdffdd-7b8a-4039-812c-557fc6c8525d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:29.824104 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:29.827104 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:29.832793 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2bd218-6dc6-4b5b-99a6-01b3fadaf1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:29.833468 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bd218-6dc6-4b5b-99a6-01b3fadaf1ab None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:29.833752 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bd218-6dc6-4b5b-99a6-01b3fadaf1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:29.834026 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bd218-6dc6-4b5b-99a6-01b3fadaf1ab None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:29.838858 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2bd218-6dc6-4b5b-99a6-01b3fadaf1ab None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:29.838858 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2bd218-6dc6-4b5b-99a6-01b3fadaf1ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:29.838858 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2bd218-6dc6-4b5b-99a6-01b3fadaf1ab None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:29.853763 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4cb035-8663-46e8-9249-5cc87815de9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:29.854781 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4cb035-8663-46e8-9249-5cc87815de9d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:29.855308 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4cb035-8663-46e8-9249-5cc87815de9d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:29.855585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 342/692] 10.222.0.24 () {56 vars in 1133 bytes} [Fri May 27 03:15:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:15:29.855993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4cb035-8663-46e8-9249-5cc87815de9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:29.861127 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2bd218-6dc6-4b5b-99a6-01b3fadaf1ab None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:29.990167 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4cb035-8663-46e8-9249-5cc87815de9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:29.992022 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4cb035-8663-46e8-9249-5cc87815de9d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:30.044135 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2bd218-6dc6-4b5b-99a6-01b3fadaf1ab None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:30.100616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a552eec-577a-4b0d-a2ce-6f863c0a617d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:30.101659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a552eec-577a-4b0d-a2ce-6f863c0a617d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:30.102155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a552eec-577a-4b0d-a2ce-6f863c0a617d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:30.102624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a552eec-577a-4b0d-a2ce-6f863c0a617d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:30.190208 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 351/693] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:15:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 621 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:15:30.197581 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4cb035-8663-46e8-9249-5cc87815de9d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:30.216357 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:30.222033 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:30.247281 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a552eec-577a-4b0d-a2ce-6f863c0a617d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:30.248742 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a552eec-577a-4b0d-a2ce-6f863c0a617d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:30.309062 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4cb035-8663-46e8-9249-5cc87815de9d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:30.355243 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7973083-7bc4-48ac-962c-5fca355046be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:30.356343 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7973083-7bc4-48ac-962c-5fca355046be None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:30.356629 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7973083-7bc4-48ac-962c-5fca355046be None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:30.356906 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7973083-7bc4-48ac-962c-5fca355046be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:30.369203 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7973083-7bc4-48ac-962c-5fca355046be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:30.370894 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7973083-7bc4-48ac-962c-5fca355046be None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:30.393021 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-a7973083-7bc4-48ac-962c-5fca355046be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:30.393021 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:30.393021 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:30.393021 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:30.393021 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:30.393021 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:30.393021 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:30.393021 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:30.393021 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:30.393021 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:30.393021 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:30.393021 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:30.393021 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:30.393021 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:30.393021 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:30.393954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:30.393954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:30.393954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:30.393954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:30.393954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:30.393954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:30.393954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:30.393954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:30.393954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:30.393954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:30.393954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:30.393954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:30.393954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:30.393954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:30.393954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:30.393954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:30.393954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:30.394825 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:30.394825 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:30.394825 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:30.394825 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:30.394825 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 352/694] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:15:30.400389 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7973083-7bc4-48ac-962c-5fca355046be None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:30.469918 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 343/695] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:15:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:15:30.471372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967eb4c-428c-40ba-80e4-b1b550a3ae8c None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:30.471665 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967eb4c-428c-40ba-80e4-b1b550a3ae8c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:30.471944 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967eb4c-428c-40ba-80e4-b1b550a3ae8c None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:30.473465 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 344/696] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:30.474172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e967eb4c-428c-40ba-80e4-b1b550a3ae8c None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:30.496104 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:30.528321 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7973083-7bc4-48ac-962c-5fca355046be None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:30.565617 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337fd7a7-d5a1-43ab-be59-35df5762800b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:30.566643 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fd7a7-d5a1-43ab-be59-35df5762800b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:30.566643 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fd7a7-d5a1-43ab-be59-35df5762800b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:30.566822 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fd7a7-d5a1-43ab-be59-35df5762800b None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:30.567465 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337fd7a7-d5a1-43ab-be59-35df5762800b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:30.567769 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337fd7a7-d5a1-43ab-be59-35df5762800b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:30.569125 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337fd7a7-d5a1-43ab-be59-35df5762800b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:30.580795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:30.582874 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 353/697] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:30.582874 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb1682-a01e-47d5-8409-b30b53523b01 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:30.582874 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb1682-a01e-47d5-8409-b30b53523b01 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:30.583211 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb1682-a01e-47d5-8409-b30b53523b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:30.601202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-86fb1682-a01e-47d5-8409-b30b53523b01 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:30.602434 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-86fb1682-a01e-47d5-8409-b30b53523b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:30.612861 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e967eb4c-428c-40ba-80e4-b1b550a3ae8c None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:30.660086 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e9722-1fe2-4c38-b30c-9766e4a9b4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:30.661350 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e9722-1fe2-4c38-b30c-9766e4a9b4de None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:30.661636 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e9722-1fe2-4c38-b30c-9766e4a9b4de None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:30.661914 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e9722-1fe2-4c38-b30c-9766e4a9b4de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:30.674619 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e9722-1fe2-4c38-b30c-9766e4a9b4de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:30.677057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e9722-1fe2-4c38-b30c-9766e4a9b4de None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:30.686929 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 354/698] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:15:30.687813 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb1682-a01e-47d5-8409-b30b53523b01 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:30.704144 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-1c2e9722-1fe2-4c38-b30c-9766e4a9b4de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:30.704144 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:30.704144 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:30.704144 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:30.704144 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:30.704144 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:30.704144 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:30.704144 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:30.704144 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:30.704144 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:30.704144 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:30.704144 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:30.704144 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:30.704144 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:30.704144 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:30.705616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:30.705616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:30.705616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:30.705616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:30.705616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:30.705616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:30.705616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:30.705616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:30.705616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:30.705616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:30.705616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:30.705616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:30.705616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:30.705616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:30.705616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:30.705616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:30.705616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:30.706710 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:30.706710 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:30.706710 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:30.706710 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:30.706710 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 345/699] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:30.718344 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e9722-1fe2-4c38-b30c-9766e4a9b4de None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:30.766858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:30.808978 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb1682-a01e-47d5-8409-b30b53523b01 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:30.831754 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e9722-1fe2-4c38-b30c-9766e4a9b4de None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:30.843614 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0e8f9-3b77-4900-aebc-64f1232a73e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:30.844406 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0e8f9-3b77-4900-aebc-64f1232a73e2 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:30.844688 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0e8f9-3b77-4900-aebc-64f1232a73e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:30.845128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0e8f9-3b77-4900-aebc-64f1232a73e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:30.851806 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc0e8f9-3b77-4900-aebc-64f1232a73e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:30.853296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc0e8f9-3b77-4900-aebc-64f1232a73e2 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:30.866083 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:30.867224 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-adc0e8f9-3b77-4900-aebc-64f1232a73e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:30.867224 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:30.867224 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:30.867224 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:30.867224 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:30.867224 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:30.867224 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:30.867224 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:30.867224 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:30.867224 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:30.867224 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:30.867224 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:30.867224 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:15:30.867224 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:15:30.867224 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:15:30.868099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:15:30.868099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:30.868099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:30.868099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:30.868099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:30.868099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:30.868099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:30.868099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:30.868099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:30.868099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:30.868099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:30.868099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:30.868099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:30.868099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:30.868099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:30.868099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:30.868099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:30.869008 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:15:30.869008 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:15:30.869008 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:15:30.869008 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:15:30.869008 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:15:30.869008 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:15:30.869008 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:30.869008 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:30.869008 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 355/700] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:30.869485 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0e8f9-3b77-4900-aebc-64f1232a73e2 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:30.875201 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1320b-123c-46af-959e-a48b74aa15ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:30.876223 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1320b-123c-46af-959e-a48b74aa15ae None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:30.876700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1320b-123c-46af-959e-a48b74aa15ae None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:30.877138 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1320b-123c-46af-959e-a48b74aa15ae None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:30.878023 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1320b-123c-46af-959e-a48b74aa15ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:30.878608 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1320b-123c-46af-959e-a48b74aa15ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:30.880291 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1320b-123c-46af-959e-a48b74aa15ae None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:30.898640 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 346/701] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:30.899912 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1320b-123c-46af-959e-a48b74aa15ae None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:30.976570 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0e8f9-3b77-4900-aebc-64f1232a73e2 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:31.001751 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:31.008575 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1320b-123c-46af-959e-a48b74aa15ae None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:31.013914 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0ed2f-a4fb-4129-a4c5-3c98537c1ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:31.014699 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0ed2f-a4fb-4129-a4c5-3c98537c1ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:31.015010 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0ed2f-a4fb-4129-a4c5-3c98537c1ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:31.015284 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0ed2f-a4fb-4129-a4c5-3c98537c1ee9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:31.027332 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b0ed2f-a4fb-4129-a4c5-3c98537c1ee9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:31.028941 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b0ed2f-a4fb-4129-a4c5-3c98537c1ee9 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:31.048364 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262cbb3-b5e9-4bef-9336-4dbb2d57c4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:31.048662 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-d3b0ed2f-a4fb-4129-a4c5-3c98537c1ee9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:31.048662 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:31.048662 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:31.048662 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:31.048662 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:31.048662 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:31.048662 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:31.048662 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:31.048662 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:31.048662 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:31.048662 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:31.048662 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:31.048662 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:31.048662 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:31.048662 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:31.049514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:31.049514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:31.049514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:31.049514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:31.049514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:31.049514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:31.049514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:31.049514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:31.049514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:31.049514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:31.049514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:31.049514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:31.049514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:31.049514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:31.049514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:31.049514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:31.049514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:31.050424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:31.050424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:31.050424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:31.050424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:31.052020 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262cbb3-b5e9-4bef-9336-4dbb2d57c4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:31.052504 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262cbb3-b5e9-4bef-9336-4dbb2d57c4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:31.052985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262cbb3-b5e9-4bef-9336-4dbb2d57c4a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:31.053991 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262cbb3-b5e9-4bef-9336-4dbb2d57c4a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:31.054534 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 356/702] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:31.055152 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262cbb3-b5e9-4bef-9336-4dbb2d57c4a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:31.055406 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0ed2f-a4fb-4129-a4c5-3c98537c1ee9 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:31.057242 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262cbb3-b5e9-4bef-9336-4dbb2d57c4a2 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:31.070683 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 347/703] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:15:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:31.072487 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262cbb3-b5e9-4bef-9336-4dbb2d57c4a2 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:31.130712 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:31.173071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0ed2f-a4fb-4129-a4c5-3c98537c1ee9 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:31.182157 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262cbb3-b5e9-4bef-9336-4dbb2d57c4a2 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:31.210077 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9cb77-b157-46e2-8efe-482e2e4ee5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:31.211228 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9cb77-b157-46e2-8efe-482e2e4ee5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:31.211767 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9cb77-b157-46e2-8efe-482e2e4ee5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:31.212326 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9cb77-b157-46e2-8efe-482e2e4ee5d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:31.219595 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9cb77-b157-46e2-8efe-482e2e4ee5d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:31.219985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869fe12c-73ce-4e25-9a2b-90069446c742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:31.220653 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fe12c-73ce-4e25-9a2b-90069446c742 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:31.220933 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fe12c-73ce-4e25-9a2b-90069446c742 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:31.221208 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fe12c-73ce-4e25-9a2b-90069446c742 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:31.221931 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869fe12c-73ce-4e25-9a2b-90069446c742 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:31.222238 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869fe12c-73ce-4e25-9a2b-90069446c742 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:31.222555 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9cb77-b157-46e2-8efe-482e2e4ee5d1 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:31.223821 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869fe12c-73ce-4e25-9a2b-90069446c742 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:31.240773 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 348/704] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:31.240773 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869fe12c-73ce-4e25-9a2b-90069446c742 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:31.244192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-1af9cb77-b157-46e2-8efe-482e2e4ee5d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:31.244192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:31.244192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:31.244192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:31.244192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:31.244192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:31.244192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:31.244192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:31.244192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:31.244192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:31.244192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:31.244192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:31.244192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:15:31.244192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:15:31.244192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:15:31.245038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:15:31.245038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:31.245038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:31.245038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:31.245038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:31.245038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:31.245038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:31.245038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:31.245038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:31.245038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:31.245038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:31.245038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:31.245038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:31.245038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:31.245038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:31.245038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:31.245038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:31.245924 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:15:31.245924 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:15:31.245924 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:15:31.245924 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:15:31.245924 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:15:31.245924 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:15:31.245924 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:31.245924 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:31.248421 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 357/705] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:31.249476 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9cb77-b157-46e2-8efe-482e2e4ee5d1 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:31.289687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:31.358212 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869fe12c-73ce-4e25-9a2b-90069446c742 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:31.358886 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9cb77-b157-46e2-8efe-482e2e4ee5d1 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:31.375389 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:31.398360 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702abe34-c612-40a9-80d4-73a53d3c6a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:31.399005 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702abe34-c612-40a9-80d4-73a53d3c6a08 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:31.399287 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702abe34-c612-40a9-80d4-73a53d3c6a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:31.399572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702abe34-c612-40a9-80d4-73a53d3c6a08 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:31.400906 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702abe34-c612-40a9-80d4-73a53d3c6a08 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:31.401589 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ab269-c5aa-46e7-84c8-7780c245d8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:31.402328 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702abe34-c612-40a9-80d4-73a53d3c6a08 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:31.402673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ab269-c5aa-46e7-84c8-7780c245d8df None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:31.403169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ab269-c5aa-46e7-84c8-7780c245d8df None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:31.403771 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ab269-c5aa-46e7-84c8-7780c245d8df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:31.404799 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ab269-c5aa-46e7-84c8-7780c245d8df None admin] RBAC: Adding query filter params (role.id=8e315ffd561a48f0891ef53b51d548d8, scope.project.id=6ceffdd955324aec84ba048d2137ae00, user.id=21d559cd7127490da84185f20684929d) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:31.405311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ab269-c5aa-46e7-84c8-7780c245d8df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:31.406963 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ab269-c5aa-46e7-84c8-7780c245d8df None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:31.414401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 349/706] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:15:31 2022] GET /identity/v3/role_assignments?user.id=21d559cd7127490da84185f20684929d&scope.project.id=6ceffdd955324aec84ba048d2137ae00&role.id=8e315ffd561a48f0891ef53b51d548d8 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:31.416163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ab269-c5aa-46e7-84c8-7780c245d8df None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:31.498880 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 358/707] 10.222.0.24 () {60 vars in 1132 bytes} [Fri May 27 03:15:31 2022] POST /identity/v3/users => generated 254 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:15:31.550871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ab269-c5aa-46e7-84c8-7780c245d8df None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:31.558111 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6e3da-367e-40d7-8030-279641abce37 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:31.558482 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6e3da-367e-40d7-8030-279641abce37 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:31.558660 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6e3da-367e-40d7-8030-279641abce37 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:31.560534 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 359/708] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:31.566531 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8740ca-a17b-4cef-9f5a-4032f2ca498d None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:31.566879 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8740ca-a17b-4cef-9f5a-4032f2ca498d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:31.567112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8740ca-a17b-4cef-9f5a-4032f2ca498d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:31.595760 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556e6db-2894-4e31-b09e-89a72ea0170e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:31.596838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556e6db-2894-4e31-b09e-89a72ea0170e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:31.597324 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556e6db-2894-4e31-b09e-89a72ea0170e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:31.597795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556e6db-2894-4e31-b09e-89a72ea0170e None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:31.598800 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d556e6db-2894-4e31-b09e-89a72ea0170e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:31.599357 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d556e6db-2894-4e31-b09e-89a72ea0170e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:31.601256 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d556e6db-2894-4e31-b09e-89a72ea0170e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:31.604445 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-9c8740ca-a17b-4cef-9f5a-4032f2ca498d None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:31.605735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-9c8740ca-a17b-4cef-9f5a-4032f2ca498d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:31.617381 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 350/709] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:15:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:31.623817 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556e6db-2894-4e31-b09e-89a72ea0170e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:31.666653 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:31.720748 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 360/710] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:15:31.744973 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8740ca-a17b-4cef-9f5a-4032f2ca498d None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:31.750533 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d556e6db-2894-4e31-b09e-89a72ea0170e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:31.773108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:31.789598 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74343d43-0f4e-427b-83fa-a7ee1b3e7c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:31.800721 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74343d43-0f4e-427b-83fa-a7ee1b3e7c4d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:31.800721 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74343d43-0f4e-427b-83fa-a7ee1b3e7c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:31.800721 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74343d43-0f4e-427b-83fa-a7ee1b3e7c4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:31.800721 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74343d43-0f4e-427b-83fa-a7ee1b3e7c4d None admin] RBAC: Adding query filter params (role.id=e5a908bd3ed5445b82a6053246533b88, scope.project.id=6ceffdd955324aec84ba048d2137ae00, user.id=22a9592143014f3287cd75c60ca17237) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:31.800721 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74343d43-0f4e-427b-83fa-a7ee1b3e7c4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:31.800721 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74343d43-0f4e-427b-83fa-a7ee1b3e7c4d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:31.802172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 351/711] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:15:31 2022] GET /identity/v3/role_assignments?user.id=22a9592143014f3287cd75c60ca17237&scope.project.id=6ceffdd955324aec84ba048d2137ae00&role.id=e5a908bd3ed5445b82a6053246533b88 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:31.802172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7423156-d168-46cc-9dd5-7ea274dac60b None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:31.802172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7423156-d168-46cc-9dd5-7ea274dac60b None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:31.802172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7423156-d168-46cc-9dd5-7ea274dac60b None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:31.802172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 352/712] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:31.808546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd45c59-98e4-46c5-9c62-71d0bab9d6ce None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:31.808828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd45c59-98e4-46c5-9c62-71d0bab9d6ce None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:31.809105 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd45c59-98e4-46c5-9c62-71d0bab9d6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:31.828075 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-2cd45c59-98e4-46c5-9c62-71d0bab9d6ce None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:31.829594 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-2cd45c59-98e4-46c5-9c62-71d0bab9d6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:31.892724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8740ca-a17b-4cef-9f5a-4032f2ca498d None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:31.934287 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 353/713] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:15:31.951633 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89839126-5094-4c51-9d4a-a2af338a7259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:31.952773 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89839126-5094-4c51-9d4a-a2af338a7259 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:31.953203 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89839126-5094-4c51-9d4a-a2af338a7259 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:31.953617 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89839126-5094-4c51-9d4a-a2af338a7259 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:31.954436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89839126-5094-4c51-9d4a-a2af338a7259 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:31.956220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89839126-5094-4c51-9d4a-a2af338a7259 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:31.972959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-89839126-5094-4c51-9d4a-a2af338a7259 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:15:31.972959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:31.972959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:15:31.972959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:15:31.972959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:15:31.972959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:15:31.972959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:15:31.972959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:15:31.972959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:15:31.972959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:31.972959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:15:31.972959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:31.972959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:31.972959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:31.972959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:31.972959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:31.974299 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:31.974299 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:31.974299 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:31.974299 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:31.974299 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:31.974299 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:31.974299 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:31.974299 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:15:31.974299 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:15:31.974299 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:15:31.974299 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:15:31.974299 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:31.974299 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:31.974299 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:31.974299 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:31.974299 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:31.975355 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:31.975355 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:31.975355 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:31.975355 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:31.975355 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:31.975355 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:31.975355 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:31.975355 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:31.975355 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:31.975355 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:31.975355 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:31.975355 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:15:31.975355 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:15:31.975355 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:15:31.975355 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:31.978143 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 361/714] 10.222.0.24 () {56 vars in 1129 bytes} [Fri May 27 03:15:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:15:31.983984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89839126-5094-4c51-9d4a-a2af338a7259 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:31.984147 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd45c59-98e4-46c5-9c62-71d0bab9d6ce None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:32.095202 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:32.222381 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:32.245246 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd45c59-98e4-46c5-9c62-71d0bab9d6ce None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:32.255982 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89839126-5094-4c51-9d4a-a2af338a7259 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:32.296296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c30d753-4cbc-454b-906d-e9fc5bec88f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:32.296966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30d753-4cbc-454b-906d-e9fc5bec88f2 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:32.297245 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30d753-4cbc-454b-906d-e9fc5bec88f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:32.297517 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30d753-4cbc-454b-906d-e9fc5bec88f2 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:32.298230 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c30d753-4cbc-454b-906d-e9fc5bec88f2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:32.298535 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c30d753-4cbc-454b-906d-e9fc5bec88f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:32.299938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c30d753-4cbc-454b-906d-e9fc5bec88f2 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:32.306942 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 354/715] 10.222.0.24 () {56 vars in 1124 bytes} [Fri May 27 03:15:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:32.315830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c30d753-4cbc-454b-906d-e9fc5bec88f2 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:32.342502 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ccb99-17b0-4e20-a509-bd8e3c8c8bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:32.343477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ccb99-17b0-4e20-a509-bd8e3c8c8bbd None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:32.343761 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ccb99-17b0-4e20-a509-bd8e3c8c8bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:32.344038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6ccb99-17b0-4e20-a509-bd8e3c8c8bbd None admin] PATH_INFO: `/v3/services/network` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:32.344703 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ccb99-17b0-4e20-a509-bd8e3c8c8bbd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:32.346294 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6ccb99-17b0-4e20-a509-bd8e3c8c8bbd None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:32.365456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-cb6ccb99-17b0-4e20-a509-bd8e3c8c8bbd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:15:32.365456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:32.365456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:15:32.365456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:15:32.365456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:15:32.365456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:15:32.365456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:15:32.365456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:15:32.365456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:15:32.365456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:32.365456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:15:32.365456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:32.365456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:32.365456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:32.365456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:32.365456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:32.365456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:32.366539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:32.366539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:32.366539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:32.366539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:32.366539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:32.366539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:32.366539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:15:32.366539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:15:32.366539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:15:32.366539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:15:32.366539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:32.366539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:32.366539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:32.366539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:32.366539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:32.366539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:32.367450 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:32.367450 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:32.367450 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:32.367450 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:32.367450 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:32.367450 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:32.367450 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:32.367450 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:32.367450 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:32.367450 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:32.367450 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:15:32.367450 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:15:32.367450 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:15:32.367450 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:32.367450 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 362/716] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:31 2022] GET /identity/v3/services/network => generated 88 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:15:32.383585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ccb99-17b0-4e20-a509-bd8e3c8c8bbd None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:32.423155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:32.430067 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c30d753-4cbc-454b-906d-e9fc5bec88f2 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:32.465041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5defb-935d-4851-bddc-e650b7ca28cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:32.465717 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5defb-935d-4851-bddc-e650b7ca28cf None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:32.466002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5defb-935d-4851-bddc-e650b7ca28cf None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:32.466285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff5defb-935d-4851-bddc-e650b7ca28cf None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:32.467859 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5defb-935d-4851-bddc-e650b7ca28cf None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:32.468182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5defb-935d-4851-bddc-e650b7ca28cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:32.473939 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff5defb-935d-4851-bddc-e650b7ca28cf None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:32.484051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 355/717] 10.222.0.24 () {56 vars in 1124 bytes} [Fri May 27 03:15:32 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:32.584056 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6ccb99-17b0-4e20-a509-bd8e3c8c8bbd None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:32.639533 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a0739f-6e75-4d31-bad0-6c2fe8b6679b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:32.640206 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0739f-6e75-4d31-bad0-6c2fe8b6679b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:32.640489 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0739f-6e75-4d31-bad0-6c2fe8b6679b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:32.640765 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0739f-6e75-4d31-bad0-6c2fe8b6679b None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:32.641479 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a0739f-6e75-4d31-bad0-6c2fe8b6679b None admin] RBAC: Adding query filter params (name=network) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:32.641786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a0739f-6e75-4d31-bad0-6c2fe8b6679b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:32.643177 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a0739f-6e75-4d31-bad0-6c2fe8b6679b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:32.652252 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 363/718] 10.222.0.24 () {56 vars in 1112 bytes} [Fri May 27 03:15:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:15:32.653535 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5defb-935d-4851-bddc-e650b7ca28cf None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:32.783327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff5defb-935d-4851-bddc-e650b7ca28cf None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:32.817900 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff850c2b-a145-438e-b0a1-09aaa0273b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:32.818577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff850c2b-a145-438e-b0a1-09aaa0273b6c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:32.818874 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff850c2b-a145-438e-b0a1-09aaa0273b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:32.819161 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff850c2b-a145-438e-b0a1-09aaa0273b6c None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:32.819872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff850c2b-a145-438e-b0a1-09aaa0273b6c None admin] RBAC: Adding query filter params (type=network) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:32.820182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff850c2b-a145-438e-b0a1-09aaa0273b6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:32.821578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff850c2b-a145-438e-b0a1-09aaa0273b6c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:32.826707 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 356/719] 10.222.0.24 () {56 vars in 1112 bytes} [Fri May 27 03:15:32 2022] GET /identity/v3/services?type=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:34.280940 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd5d8b-372a-4594-b6ac-d58cc12dfa3d None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:34.281451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd5d8b-372a-4594-b6ac-d58cc12dfa3d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:34.281527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd5d8b-372a-4594-b6ac-d58cc12dfa3d None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:34.283152 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 364/720] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:34.287267 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182f7eb-349e-4bea-a7e5-459b33956ad8 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:34.287681 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182f7eb-349e-4bea-a7e5-459b33956ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:34.287875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182f7eb-349e-4bea-a7e5-459b33956ad8 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:34.289379 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 357/721] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:34.290669 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13016445-9145-46bf-99e3-852af1a8bf31 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:34.290997 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13016445-9145-46bf-99e3-852af1a8bf31 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:34.291272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13016445-9145-46bf-99e3-852af1a8bf31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:34.294095 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71f934-5999-457a-96de-e5b8303d19a5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:34.294388 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71f934-5999-457a-96de-e5b8303d19a5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:34.294667 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71f934-5999-457a-96de-e5b8303d19a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:34.312815 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-ec71f934-5999-457a-96de-e5b8303d19a5 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:34.314011 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ec71f934-5999-457a-96de-e5b8303d19a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:34.318054 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-13016445-9145-46bf-99e3-852af1a8bf31 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:34.319267 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-13016445-9145-46bf-99e3-852af1a8bf31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:34.331798 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:34.332478 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:34.398488 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 358/722] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:15:34.399907 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb2761-83b1-4e46-a2dc-204896da4437 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:34.400196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb2761-83b1-4e46-a2dc-204896da4437 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:34.400528 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb2761-83b1-4e46-a2dc-204896da4437 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:34.402035 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 359/723] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:34.407240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085975a-2b6e-4f54-b793-5a43c548833f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:34.407523 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085975a-2b6e-4f54-b793-5a43c548833f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:34.407795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5085975a-2b6e-4f54-b793-5a43c548833f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:34.424484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-5085975a-2b6e-4f54-b793-5a43c548833f None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:34.425386 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 365/724] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:15:34.425673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-5085975a-2b6e-4f54-b793-5a43c548833f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:34.438562 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13016445-9145-46bf-99e3-852af1a8bf31 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:34.494792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 360/725] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:15:34.495668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085975a-2b6e-4f54-b793-5a43c548833f None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:34.563744 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13016445-9145-46bf-99e3-852af1a8bf31 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:34.587507 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:34.601676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e6d98-e178-4884-a45b-8352d36702d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:34.603718 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e6d98-e178-4884-a45b-8352d36702d1 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:34.603718 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e6d98-e178-4884-a45b-8352d36702d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:34.604101 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e6d98-e178-4884-a45b-8352d36702d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:34.605875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5085975a-2b6e-4f54-b793-5a43c548833f None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:34.615178 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e6d98-e178-4884-a45b-8352d36702d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:34.623281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-fd0e6d98-e178-4884-a45b-8352d36702d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:15:34.623281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:34.623281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:34.623281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:34.623281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:34.623281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:34.623281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:34.623281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:34.623281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:34.623281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:34.623281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:34.623281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:34.623281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:34.623281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:34.623281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:34.623281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:34.624202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:34.624202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:34.624202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:34.624202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:34.624202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:34.624202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:34.624202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:34.624202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:34.624202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:34.624202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:34.624202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:34.624202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:34.624202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:34.624202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:34.624202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:34.624202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:34.625032 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:34.625032 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:34.625032 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:34.625032 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:34.625032 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:34.625032 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:34.625032 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:34.625032 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:34.625032 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:34.625032 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:34.625032 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:15:34.625032 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:34.633061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 366/726] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:15:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:15:34.635160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e6d98-e178-4884-a45b-8352d36702d1 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:34.640276 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:34.642088 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ea5c0-e78e-4c49-9e40-198b699a4fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:34.642995 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ea5c0-e78e-4c49-9e40-198b699a4fba None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:34.643281 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ea5c0-e78e-4c49-9e40-198b699a4fba None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:34.643552 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ea5c0-e78e-4c49-9e40-198b699a4fba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:34.655903 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ea5c0-e78e-4c49-9e40-198b699a4fba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:34.663422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-e19ea5c0-e78e-4c49-9e40-198b699a4fba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:15:34.663422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:34.663422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:34.663422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:34.663422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:34.663422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:34.663422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:34.663422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:34.663422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:34.663422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:34.663422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:34.663422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:34.663422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:34.663422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:34.663422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:34.663422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:34.664351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:34.664351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:34.664351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:34.664351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:34.664351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:34.664351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:34.664351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:34.664351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:34.664351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:34.664351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:34.664351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:34.664351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:34.664351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:34.664351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:34.664351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:34.664351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:34.665227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:34.665227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:34.665227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:34.665227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:34.665227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:34.665227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:34.665227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:34.665227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:34.665227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:34.665227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:34.665227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:15:34.665227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:34.665227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 361/727] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:15:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:34.666410 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ea5c0-e78e-4c49-9e40-198b699a4fba None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:34.763164 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e6d98-e178-4884-a45b-8352d36702d1 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:34.770729 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ea5c0-e78e-4c49-9e40-198b699a4fba None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:34.805924 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928b0ce-d0c5-443b-8bc7-1166c3fd9659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:34.806650 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928b0ce-d0c5-443b-8bc7-1166c3fd9659 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:34.806965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928b0ce-d0c5-443b-8bc7-1166c3fd9659 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:34.807185 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22d310-756b-4eb6-8c55-af47444458d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:34.807321 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928b0ce-d0c5-443b-8bc7-1166c3fd9659 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:34.808037 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928b0ce-d0c5-443b-8bc7-1166c3fd9659 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:34.808349 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928b0ce-d0c5-443b-8bc7-1166c3fd9659 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:34.808734 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22d310-756b-4eb6-8c55-af47444458d1 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:34.809223 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22d310-756b-4eb6-8c55-af47444458d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:34.809708 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22d310-756b-4eb6-8c55-af47444458d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:34.809812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6928b0ce-d0c5-443b-8bc7-1166c3fd9659 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:34.822200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb22d310-756b-4eb6-8c55-af47444458d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:34.824000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 362/728] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:15:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:34.824892 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928b0ce-d0c5-443b-8bc7-1166c3fd9659 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:34.828708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:34.832661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-fb22d310-756b-4eb6-8c55-af47444458d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:34.832661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:34.832661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:34.832661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:34.832661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:34.832661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:34.832661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:34.832661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:34.832661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:34.832661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:34.832661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:34.832661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:34.832661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:34.832661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:34.832661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:34.832661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:34.833593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:34.833593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:34.833593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:34.833593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:34.833593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:34.833593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:34.833593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:34.833593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:34.833593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:34.833593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:34.833593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:34.833593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:34.833593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:34.833593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:34.833593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:34.833593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:34.834436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:34.834436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:34.834436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:34.834436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:34.834436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:34.834436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:34.834436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:34.834436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:34.834436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:34.834436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:34.834436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:34.834436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:34.844259 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 367/729] 10.222.0.24 () {56 vars in 1102 bytes} [Fri May 27 03:15:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:34.844259 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22d310-756b-4eb6-8c55-af47444458d1 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:34.911349 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:34.927555 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928b0ce-d0c5-443b-8bc7-1166c3fd9659 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:34.956907 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb22d310-756b-4eb6-8c55-af47444458d1 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:34.963879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286cec7-f573-490d-a03f-690b34b3ff5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:34.964615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286cec7-f573-490d-a03f-690b34b3ff5e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:34.964896 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286cec7-f573-490d-a03f-690b34b3ff5e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:34.965222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3286cec7-f573-490d-a03f-690b34b3ff5e None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:34.965983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286cec7-f573-490d-a03f-690b34b3ff5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:34.966434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286cec7-f573-490d-a03f-690b34b3ff5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:34.967858 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3286cec7-f573-490d-a03f-690b34b3ff5e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:34.976772 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 363/730] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:15:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:34.977637 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286cec7-f573-490d-a03f-690b34b3ff5e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:34.997627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a66252-9157-48c0-9f39-5e71aadc8408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:34.998714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a66252-9157-48c0-9f39-5e71aadc8408 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:35.004751 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a66252-9157-48c0-9f39-5e71aadc8408 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:35.004751 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a66252-9157-48c0-9f39-5e71aadc8408 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:35.012248 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a66252-9157-48c0-9f39-5e71aadc8408 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:35.014050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a66252-9157-48c0-9f39-5e71aadc8408 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:35.031911 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-06a66252-9157-48c0-9f39-5e71aadc8408 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:35.031911 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:35.031911 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:35.031911 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:35.031911 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:35.031911 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:35.031911 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:35.031911 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:35.031911 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:35.031911 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:35.031911 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:35.031911 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:35.031911 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:35.031911 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:35.031911 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:35.032796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:35.032796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:35.032796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:35.032796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:35.032796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:35.032796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:35.032796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:35.032796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:35.032796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:35.032796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:35.032796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:35.032796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:35.032796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:35.032796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:35.032796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:35.032796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:35.032796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:35.033676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:35.033676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:35.033676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:35.033676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:35.034470 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 368/731] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:35.039317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a66252-9157-48c0-9f39-5e71aadc8408 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:35.139497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:35.174942 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3286cec7-f573-490d-a03f-690b34b3ff5e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:35.210589 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9030b9-6b96-415e-8eba-01df11651c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:35.211314 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9030b9-6b96-415e-8eba-01df11651c66 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:35.211598 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9030b9-6b96-415e-8eba-01df11651c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:35.211994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9030b9-6b96-415e-8eba-01df11651c66 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:35.212658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9030b9-6b96-415e-8eba-01df11651c66 None admin] RBAC: Adding query filter params (name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:35.212969 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9030b9-6b96-415e-8eba-01df11651c66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:35.214343 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9030b9-6b96-415e-8eba-01df11651c66 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:35.220153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a66252-9157-48c0-9f39-5e71aadc8408 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:35.227350 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 364/732] 10.222.0.24 () {56 vars in 1103 bytes} [Fri May 27 03:15:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:35.228336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9030b9-6b96-415e-8eba-01df11651c66 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:35.239111 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:35.260866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e719851-9877-47f3-85aa-282e1cd6731b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:35.262251 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e719851-9877-47f3-85aa-282e1cd6731b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:35.262759 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e719851-9877-47f3-85aa-282e1cd6731b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:35.263270 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e719851-9877-47f3-85aa-282e1cd6731b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:35.275433 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e719851-9877-47f3-85aa-282e1cd6731b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:35.276898 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e719851-9877-47f3-85aa-282e1cd6731b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:35.295063 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-6e719851-9877-47f3-85aa-282e1cd6731b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:35.295063 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:35.295063 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:35.295063 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:35.295063 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:35.295063 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:35.295063 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:35.295063 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:35.295063 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:35.295063 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:35.295063 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:35.295063 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:35.295063 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:35.295063 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:35.295063 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:35.296392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:35.296392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:35.296392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:35.296392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:35.296392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:35.296392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:35.296392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:35.296392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:35.296392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:35.296392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:35.296392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:35.296392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:35.296392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:35.296392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:35.296392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:35.296392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:35.296392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:35.297279 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:35.297279 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:35.297279 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:35.297279 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:35.297279 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 369/733] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:35.297550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e719851-9877-47f3-85aa-282e1cd6731b None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:35.335230 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9030b9-6b96-415e-8eba-01df11651c66 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:35.362225 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:35.370385 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f967c42e-4ede-4c2d-bcfb-c87b53ce83fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:35.371073 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967c42e-4ede-4c2d-bcfb-c87b53ce83fb None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:35.383513 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967c42e-4ede-4c2d-bcfb-c87b53ce83fb None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:35.383513 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967c42e-4ede-4c2d-bcfb-c87b53ce83fb None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:35.383513 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f967c42e-4ede-4c2d-bcfb-c87b53ce83fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:35.383513 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f967c42e-4ede-4c2d-bcfb-c87b53ce83fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:35.383513 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f967c42e-4ede-4c2d-bcfb-c87b53ce83fb None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:35.391323 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 365/734] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:35.391323 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891eaed2-38e5-4f78-bc1a-c172468c3331 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:35.391323 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891eaed2-38e5-4f78-bc1a-c172468c3331 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:35.391323 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891eaed2-38e5-4f78-bc1a-c172468c3331 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:35.391323 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 366/735] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:35.391323 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891eaed2-38e5-4f78-bc1a-c172468c3331 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:35.414725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e719851-9877-47f3-85aa-282e1cd6731b None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:35.449276 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27537203-b64b-4592-a765-926b606dca5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:35.450064 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27537203-b64b-4592-a765-926b606dca5c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:35.450587 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27537203-b64b-4592-a765-926b606dca5c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:35.450862 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27537203-b64b-4592-a765-926b606dca5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:35.463009 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27537203-b64b-4592-a765-926b606dca5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:35.464476 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27537203-b64b-4592-a765-926b606dca5c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:35.483366 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-27537203-b64b-4592-a765-926b606dca5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:35.483366 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:35.483366 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:35.483366 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:35.483366 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:35.483366 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:35.483366 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:35.483366 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:35.483366 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:35.483366 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:35.483366 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:35.483366 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:35.483366 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:35.483366 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:35.483366 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:35.484370 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:35.484370 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:35.484370 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:35.484370 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:35.484370 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:35.484370 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:35.484370 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:35.484370 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:35.484370 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:35.484370 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:35.484370 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:35.484370 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:35.484370 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:35.484370 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:35.484370 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:35.484370 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:35.484370 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:35.485259 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:35.485259 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:35.485259 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:35.485259 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:35.485259 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 370/736] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:35.486106 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27537203-b64b-4592-a765-926b606dca5c None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:35.507087 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891eaed2-38e5-4f78-bc1a-c172468c3331 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:35.511202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:35.543612 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e502ee-c833-4730-97be-6a9caa8f8d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:35.544290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e502ee-c833-4730-97be-6a9caa8f8d20 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:35.544572 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e502ee-c833-4730-97be-6a9caa8f8d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:35.544853 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e502ee-c833-4730-97be-6a9caa8f8d20 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:35.545592 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e502ee-c833-4730-97be-6a9caa8f8d20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:35.545905 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e502ee-c833-4730-97be-6a9caa8f8d20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:35.547321 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e502ee-c833-4730-97be-6a9caa8f8d20 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:35.558860 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 367/737] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:35.564152 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccd90d-2442-4c97-9815-202595d3f4a5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:35.564441 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccd90d-2442-4c97-9815-202595d3f4a5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:35.564721 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccd90d-2442-4c97-9815-202595d3f4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:35.583502 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-30ccd90d-2442-4c97-9815-202595d3f4a5 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:35.584700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-30ccd90d-2442-4c97-9815-202595d3f4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:35.600331 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27537203-b64b-4592-a765-926b606dca5c None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:35.630253 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:35.639450 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43000318-8ebb-4adc-9ae2-526f4be3cc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:35.640244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43000318-8ebb-4adc-9ae2-526f4be3cc4d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:35.640525 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43000318-8ebb-4adc-9ae2-526f4be3cc4d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:35.640798 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43000318-8ebb-4adc-9ae2-526f4be3cc4d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:35.662920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43000318-8ebb-4adc-9ae2-526f4be3cc4d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:35.662920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43000318-8ebb-4adc-9ae2-526f4be3cc4d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:35.663179 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 368/738] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:15:35.663179 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccd90d-2442-4c97-9815-202595d3f4a5 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:35.664317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-43000318-8ebb-4adc-9ae2-526f4be3cc4d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:15:35.664317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:35.664317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:35.664317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:35.664317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:35.664317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:35.664317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:35.664317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:35.664317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:35.664317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:35.664317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:35.664317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:35.664317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:15:35.664317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:15:35.664317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:15:35.664317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:15:35.665972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:35.665972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:35.665972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:15:35.665972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:35.665972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:15:35.665972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:35.665972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:35.665972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:35.665972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:35.665972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:35.665972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:35.665972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:35.665972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:35.665972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:35.665972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:35.665972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:35.665972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:35.665972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:35.666972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:35.666972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:35.666972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:15:35.666972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:15:35.666972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:15:35.666972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:15:35.666972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:15:35.666972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:15:35.666972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:15:35.666972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:35.666972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 371/739] 10.222.0.24 () {56 vars in 1093 bytes} [Fri May 27 03:15:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:35.668019 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43000318-8ebb-4adc-9ae2-526f4be3cc4d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:35.750828 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:35.762192 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccd90d-2442-4c97-9815-202595d3f4a5 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:35.769105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43000318-8ebb-4adc-9ae2-526f4be3cc4d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:35.795883 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61d6ff-9e76-4cce-8e46-7ce1ca585a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:35.796541 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61d6ff-9e76-4cce-8e46-7ce1ca585a99 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:35.796833 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61d6ff-9e76-4cce-8e46-7ce1ca585a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:35.797106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61d6ff-9e76-4cce-8e46-7ce1ca585a99 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:35.797831 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e61d6ff-9e76-4cce-8e46-7ce1ca585a99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:35.798142 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e61d6ff-9e76-4cce-8e46-7ce1ca585a99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:35.799550 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e61d6ff-9e76-4cce-8e46-7ce1ca585a99 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:35.802041 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af7946-1101-443d-9c72-415924726537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:35.802887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af7946-1101-443d-9c72-415924726537 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:35.803187 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af7946-1101-443d-9c72-415924726537 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:35.803462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af7946-1101-443d-9c72-415924726537 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:35.810550 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 369/740] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:35.815043 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af7946-1101-443d-9c72-415924726537 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:35.815156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d188b-1e5a-4a74-940d-de89cc7efcd9 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:35.815156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d188b-1e5a-4a74-940d-de89cc7efcd9 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:35.815156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d188b-1e5a-4a74-940d-de89cc7efcd9 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:35.815407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af7946-1101-443d-9c72-415924726537 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:35.815491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 370/741] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:35.815491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0d188b-1e5a-4a74-940d-de89cc7efcd9 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:35.829927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-a4af7946-1101-443d-9c72-415924726537 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 03:15:35.829927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:35.829927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:35.829927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:35.829927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:35.829927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:35.829927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:35.829927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:35.829927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:35.829927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:35.829927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:35.829927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:35.829927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:15:35.829927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:15:35.829927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:15:35.829927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:15:35.830918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:35.830918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:35.830918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:15:35.830918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:35.830918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:15:35.830918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:35.830918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:35.830918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:35.830918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:35.830918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:35.830918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:35.830918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:35.830918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:35.830918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:35.830918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:35.830918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:35.830918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:35.830918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:35.831885 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:35.831885 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:35.831885 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:15:35.831885 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:15:35.831885 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:15:35.831885 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:15:35.831885 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:15:35.831885 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:15:35.831885 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 03:15:35.831885 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:35.831885 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 372/742] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:15:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:35.832470 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af7946-1101-443d-9c72-415924726537 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:35.860865 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:35.920545 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0d188b-1e5a-4a74-940d-de89cc7efcd9 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:35.931482 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af7946-1101-443d-9c72-415924726537 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:35.952655 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435cac6e-c713-419f-a83f-3a98d87e6f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:35.953315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cac6e-c713-419f-a83f-3a98d87e6f73 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:35.953598 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cac6e-c713-419f-a83f-3a98d87e6f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:35.953877 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cac6e-c713-419f-a83f-3a98d87e6f73 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:35.954705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cac6e-c713-419f-a83f-3a98d87e6f73 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:35.955043 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cac6e-c713-419f-a83f-3a98d87e6f73 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:35.956473 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cac6e-c713-419f-a83f-3a98d87e6f73 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:35.965061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f3931-8f74-4b6a-921a-5b367c168ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:35.965709 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f3931-8f74-4b6a-921a-5b367c168ece None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:35.965991 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f3931-8f74-4b6a-921a-5b367c168ece None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:35.966262 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f3931-8f74-4b6a-921a-5b367c168ece None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:35.967000 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4f3931-8f74-4b6a-921a-5b367c168ece None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:35.968337 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4f3931-8f74-4b6a-921a-5b367c168ece None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:35.975084 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 371/743] 10.222.0.24 () {56 vars in 1133 bytes} [Fri May 27 03:15:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:35.975879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435cac6e-c713-419f-a83f-3a98d87e6f73 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:36.058254 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:36.063147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 373/744] 10.222.0.24 () {60 vars in 1141 bytes} [Fri May 27 03:15:35 2022] POST /identity/v3/services => generated 251 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:15:36.064510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a9222-fcf6-4ec8-9689-1de683b119e0 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:36.064798 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a9222-fcf6-4ec8-9689-1de683b119e0 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:36.065074 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a9222-fcf6-4ec8-9689-1de683b119e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:36.081185 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:36.083136 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-d26a9222-fcf6-4ec8-9689-1de683b119e0 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:36.084235 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d26a9222-fcf6-4ec8-9689-1de683b119e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:36.089029 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435cac6e-c713-419f-a83f-3a98d87e6f73 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:36.139684 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a418b7b-0c3a-4872-a2a3-417eb32705bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:36.140670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a418b7b-0c3a-4872-a2a3-417eb32705bd None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:36.141042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a418b7b-0c3a-4872-a2a3-417eb32705bd None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:36.141409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a418b7b-0c3a-4872-a2a3-417eb32705bd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:36.151227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a418b7b-0c3a-4872-a2a3-417eb32705bd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:36.152789 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a418b7b-0c3a-4872-a2a3-417eb32705bd None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:36.171691 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 374/745] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:36 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:15:36.171691 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a9222-fcf6-4ec8-9689-1de683b119e0 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:36.171927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-7a418b7b-0c3a-4872-a2a3-417eb32705bd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:15:36.171927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:36.171927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:36.171927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:36.171927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:36.171927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:36.171927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:36.171927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:36.171927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:36.171927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:36.171927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:36.171927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:36.171927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:15:36.171927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:15:36.171927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:15:36.171927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:15:36.172871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:36.172871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:36.172871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:15:36.172871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:36.172871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:15:36.172871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:36.172871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:36.172871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:36.172871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:36.172871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:36.172871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:36.172871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:36.172871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:36.172871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:36.172871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:36.172871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:36.172871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:36.172871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:36.173828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:36.173828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:36.173828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:15:36.173828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:15:36.173828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:15:36.173828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:15:36.173828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:15:36.173828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:15:36.173828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:15:36.173828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:36.173828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 372/746] 10.222.0.24 () {56 vars in 1117 bytes} [Fri May 27 03:15:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:36.173828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a418b7b-0c3a-4872-a2a3-417eb32705bd None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:36.285071 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a418b7b-0c3a-4872-a2a3-417eb32705bd None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:36.300939 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a9222-fcf6-4ec8-9689-1de683b119e0 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:36.318716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b28db-3994-408d-afbd-3113b9ccafb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:36.319572 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b28db-3994-408d-afbd-3113b9ccafb3 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:36.319855 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b28db-3994-408d-afbd-3113b9ccafb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:36.320130 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b28db-3994-408d-afbd-3113b9ccafb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:36.331136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b28db-3994-408d-afbd-3113b9ccafb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:36.332587 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b28db-3994-408d-afbd-3113b9ccafb3 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:36.337301 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:36.343922 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b741f-189d-4ca1-8fff-8197f1913d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:36.344581 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b741f-189d-4ca1-8fff-8197f1913d9d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:36.344860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b741f-189d-4ca1-8fff-8197f1913d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:36.345135 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b741f-189d-4ca1-8fff-8197f1913d9d None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:36.345997 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b741f-189d-4ca1-8fff-8197f1913d9d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:36.346298 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b741f-189d-4ca1-8fff-8197f1913d9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:36.347734 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b741f-189d-4ca1-8fff-8197f1913d9d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:36.356300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-da4b28db-3994-408d-afbd-3113b9ccafb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:36.356300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:36.356300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:36.356300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:36.356300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:36.356300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:36.356300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:36.356300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:36.356300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:36.356300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:36.356300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:36.356300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:36.356300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:36.356300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:36.356300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:36.357474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:36.357474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:36.357474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:36.357474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:36.357474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:36.357474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:36.357474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:36.357474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:36.357474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:36.357474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:36.357474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:36.357474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:36.357474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:36.357474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:36.357474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:36.357474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:36.357474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:36.358352 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:36.358352 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:36.358352 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:36.358352 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:36.358352 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 373/747] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:36.358352 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b28db-3994-408d-afbd-3113b9ccafb3 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:36.360433 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:36.372151 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 375/748] 10.222.0.24 () {56 vars in 1135 bytes} [Fri May 27 03:15:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:36.372151 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b741f-189d-4ca1-8fff-8197f1913d9d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:36.472511 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b28db-3994-408d-afbd-3113b9ccafb3 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:36.482185 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b741f-189d-4ca1-8fff-8197f1913d9d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:36.506099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31acee-57db-482d-a111-f8741a9f3ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:36.506762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31acee-57db-482d-a111-f8741a9f3ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:36.507085 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31acee-57db-482d-a111-f8741a9f3ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:36.507359 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31acee-57db-482d-a111-f8741a9f3ee4 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:36.508199 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31acee-57db-482d-a111-f8741a9f3ee4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:36.508511 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31acee-57db-482d-a111-f8741a9f3ee4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:36.510012 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31acee-57db-482d-a111-f8741a9f3ee4 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:36.516914 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30816202-44a6-450f-a7da-e79bc469d166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:36.517572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30816202-44a6-450f-a7da-e79bc469d166 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:36.517852 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30816202-44a6-450f-a7da-e79bc469d166 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:36.518131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30816202-44a6-450f-a7da-e79bc469d166 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:36.518864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30816202-44a6-450f-a7da-e79bc469d166 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:36.520216 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30816202-44a6-450f-a7da-e79bc469d166 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:36.527870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 374/749] 10.222.0.24 () {56 vars in 1149 bytes} [Fri May 27 03:15:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:36.528689 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31acee-57db-482d-a111-f8741a9f3ee4 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:36.589119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:36.610867 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 376/750] 10.222.0.24 () {60 vars in 1141 bytes} [Fri May 27 03:15:36 2022] POST /identity/v3/services => generated 240 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:15:36.610867 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30816202-44a6-450f-a7da-e79bc469d166 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:36.630034 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31acee-57db-482d-a111-f8741a9f3ee4 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:36.676336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c96b34-a5bb-4fe9-a7f9-1d82eedb7200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:36.677045 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c96b34-a5bb-4fe9-a7f9-1d82eedb7200 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:36.677408 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c96b34-a5bb-4fe9-a7f9-1d82eedb7200 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:36.677685 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c96b34-a5bb-4fe9-a7f9-1d82eedb7200 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:36.678431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c96b34-a5bb-4fe9-a7f9-1d82eedb7200 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:36.678741 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c96b34-a5bb-4fe9-a7f9-1d82eedb7200 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:36.680330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c96b34-a5bb-4fe9-a7f9-1d82eedb7200 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:36.687513 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:36.695871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 375/751] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:36.696784 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c96b34-a5bb-4fe9-a7f9-1d82eedb7200 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:36.759896 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30816202-44a6-450f-a7da-e79bc469d166 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:36.808707 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a333263-d6d7-4f30-b2f8-a98b2f98b676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:36.808707 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a333263-d6d7-4f30-b2f8-a98b2f98b676 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:36.809012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a333263-d6d7-4f30-b2f8-a98b2f98b676 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:36.809255 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a333263-d6d7-4f30-b2f8-a98b2f98b676 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:36.813009 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c96b34-a5bb-4fe9-a7f9-1d82eedb7200 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:36.833204 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:36.853802 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe8f63-a7f7-4424-9559-cb1ba100fbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:36.866552 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe8f63-a7f7-4424-9559-cb1ba100fbf9 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:36.866552 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe8f63-a7f7-4424-9559-cb1ba100fbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:36.866552 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe8f63-a7f7-4424-9559-cb1ba100fbf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:36.893623 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a333263-d6d7-4f30-b2f8-a98b2f98b676 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:36.895452 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a333263-d6d7-4f30-b2f8-a98b2f98b676 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:36.932034 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe8f63-a7f7-4424-9559-cb1ba100fbf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:36.933499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfe8f63-a7f7-4424-9559-cb1ba100fbf9 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:36.944672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:36.997979 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 377/752] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:15:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:15:36.999178 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a333263-d6d7-4f30-b2f8-a98b2f98b676 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:37.047936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 376/753] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:15:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:15:37.048927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe8f63-a7f7-4424-9559-cb1ba100fbf9 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:37.069503 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:37.136294 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a333263-d6d7-4f30-b2f8-a98b2f98b676 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:37.169199 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfe8f63-a7f7-4424-9559-cb1ba100fbf9 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:37.183693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3979f0e-369d-476b-b788-6a3f23882629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:37.185225 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3979f0e-369d-476b-b788-6a3f23882629 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:37.185764 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3979f0e-369d-476b-b788-6a3f23882629 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:37.186191 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3979f0e-369d-476b-b788-6a3f23882629 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:37.195613 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3979f0e-369d-476b-b788-6a3f23882629 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:37.198601 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3979f0e-369d-476b-b788-6a3f23882629 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:37.205338 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1ed04-3b41-48ed-90ab-fcd825afd6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:37.206318 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1ed04-3b41-48ed-90ab-fcd825afd6cb None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:37.206699 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1ed04-3b41-48ed-90ab-fcd825afd6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:37.207199 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1ed04-3b41-48ed-90ab-fcd825afd6cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:37.219057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-d3979f0e-369d-476b-b788-6a3f23882629 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:37.219057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:37.219057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:37.219057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:37.219057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:37.219057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:37.219057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:37.219057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:37.219057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:37.219057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:37.219057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:37.219057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:37.219057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:15:37.219057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:15:37.219057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:15:37.220016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:15:37.220016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:37.220016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:37.220016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:37.220016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:37.220016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:37.220016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:37.220016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:37.220016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:37.220016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:37.220016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:37.220016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:37.220016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:37.220016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:37.220016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:37.220016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:37.220016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:37.220893 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:15:37.220893 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:15:37.220893 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:15:37.220893 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:15:37.220893 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:15:37.220893 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:15:37.220893 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:37.220893 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:37.220893 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 378/754] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:37.221367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3979f0e-369d-476b-b788-6a3f23882629 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:37.221442 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f1ed04-3b41-48ed-90ab-fcd825afd6cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:37.226865 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f1ed04-3b41-48ed-90ab-fcd825afd6cb None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:37.238450 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:37.245216 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-e5f1ed04-3b41-48ed-90ab-fcd825afd6cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:37.245216 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:37.245216 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:37.245216 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:37.245216 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:37.245216 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:37.245216 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:37.245216 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:37.245216 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:37.245216 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:37.245216 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:37.245216 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:37.245216 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:37.245216 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:37.245216 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:37.246160 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:37.246160 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:37.246160 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:37.246160 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:37.246160 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:37.246160 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:37.246160 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:37.246160 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:37.246160 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:37.246160 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:37.246160 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:37.246160 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:37.246160 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:37.246160 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:37.246160 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:37.246160 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:37.246160 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:37.252331 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:37.252331 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:37.252331 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:37.252331 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:37.252331 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 377/755] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:37.252331 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1ed04-3b41-48ed-90ab-fcd825afd6cb None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:37.337287 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:37.347925 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3979f0e-369d-476b-b788-6a3f23882629 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:37.374041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1ed04-3b41-48ed-90ab-fcd825afd6cb None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:37.390710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a5f22c-e953-4b3d-88b9-35ff0326fedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:37.392054 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5f22c-e953-4b3d-88b9-35ff0326fedc None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:37.392693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5f22c-e953-4b3d-88b9-35ff0326fedc None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:37.393065 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5f22c-e953-4b3d-88b9-35ff0326fedc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:37.406670 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a5f22c-e953-4b3d-88b9-35ff0326fedc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:37.408708 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a5f22c-e953-4b3d-88b9-35ff0326fedc None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:37.413650 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42549d-8f49-40ec-80b2-f138fc548d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:37.414311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42549d-8f49-40ec-80b2-f138fc548d88 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:37.414595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42549d-8f49-40ec-80b2-f138fc548d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:37.414939 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42549d-8f49-40ec-80b2-f138fc548d88 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:37.415735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42549d-8f49-40ec-80b2-f138fc548d88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:37.416460 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42549d-8f49-40ec-80b2-f138fc548d88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:37.417891 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b42549d-8f49-40ec-80b2-f138fc548d88 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:37.433789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-71a5f22c-e953-4b3d-88b9-35ff0326fedc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:37.433789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:37.433789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:37.433789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:37.433789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:37.433789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:37.433789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:37.433789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:37.433789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:37.433789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:37.433789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:37.433789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:37.433789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:37.433789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:37.433789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:37.435412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:37.435412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:37.435412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:37.435412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:37.435412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:37.435412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:37.435412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:37.435412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:37.435412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:37.435412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:37.435412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:37.435412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:37.435412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:37.435412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:37.435412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:37.435412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:37.435412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:37.436653 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:37.436653 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:37.436653 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:37.436653 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:37.436919 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 378/756] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:15:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:37.436919 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42549d-8f49-40ec-80b2-f138fc548d88 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:37.437077 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 379/757] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:37.439643 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a5f22c-e953-4b3d-88b9-35ff0326fedc None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:37.507071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:37.543060 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42549d-8f49-40ec-80b2-f138fc548d88 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:37.553757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a5f22c-e953-4b3d-88b9-35ff0326fedc None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:37.573187 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:37.579455 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c8275-d1d0-422f-940a-7a9f5f4e3658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:37.580121 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c8275-d1d0-422f-940a-7a9f5f4e3658 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:37.580397 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c8275-d1d0-422f-940a-7a9f5f4e3658 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:37.580675 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4c8275-d1d0-422f-940a-7a9f5f4e3658 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:37.581391 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c8275-d1d0-422f-940a-7a9f5f4e3658 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:37.581693 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c8275-d1d0-422f-940a-7a9f5f4e3658 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:37.583355 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4c8275-d1d0-422f-940a-7a9f5f4e3658 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:37.594238 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37529a9-5148-4b01-ac9c-2898c51d341d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:37.595444 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37529a9-5148-4b01-ac9c-2898c51d341d None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:37.595766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 379/758] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:37.596215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37529a9-5148-4b01-ac9c-2898c51d341d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:37.596638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c8275-d1d0-422f-940a-7a9f5f4e3658 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:37.596945 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37529a9-5148-4b01-ac9c-2898c51d341d None admin] PATH_INFO: `/v3/projects/6ceffdd955324aec84ba048d2137ae00/users/22a9592143014f3287cd75c60ca17237/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:37.599145 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37529a9-5148-4b01-ac9c-2898c51d341d None admin] RBAC: Authorizing `identity:create_grant(project_id=6ceffdd955324aec84ba048d2137ae00, user_id=22a9592143014f3287cd75c60ca17237, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:37.600846 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37529a9-5148-4b01-ac9c-2898c51d341d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:37.671054 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 380/759] 10.222.0.24 () {58 vars in 1439 bytes} [Fri May 27 03:15:37 2022] PUT /identity/v3/projects/6ceffdd955324aec84ba048d2137ae00/users/22a9592143014f3287cd75c60ca17237/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:15:37.672493 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37529a9-5148-4b01-ac9c-2898c51d341d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:37.700248 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4c8275-d1d0-422f-940a-7a9f5f4e3658 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:37.737266 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6749e95-013a-49b8-8e3f-7d22c29e2854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:37.738039 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6749e95-013a-49b8-8e3f-7d22c29e2854 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:37.738415 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6749e95-013a-49b8-8e3f-7d22c29e2854 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:37.739027 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6749e95-013a-49b8-8e3f-7d22c29e2854 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:37.739738 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6749e95-013a-49b8-8e3f-7d22c29e2854 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:37.740568 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6749e95-013a-49b8-8e3f-7d22c29e2854 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:37.741707 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6749e95-013a-49b8-8e3f-7d22c29e2854 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:37.755716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 380/760] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:37.761117 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6749e95-013a-49b8-8e3f-7d22c29e2854 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:37.813996 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:37.872526 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6749e95-013a-49b8-8e3f-7d22c29e2854 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:37.872712 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37529a9-5148-4b01-ac9c-2898c51d341d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:37.909670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872e9f2-cde6-4913-a463-d8e11c0b3c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:37.910750 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872e9f2-cde6-4913-a463-d8e11c0b3c76 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:37.911256 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872e9f2-cde6-4913-a463-d8e11c0b3c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:37.911629 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872e9f2-cde6-4913-a463-d8e11c0b3c76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:37.914717 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:37.921442 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6872e9f2-cde6-4913-a463-d8e11c0b3c76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:37.923384 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6872e9f2-cde6-4913-a463-d8e11c0b3c76 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:37.939703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-6872e9f2-cde6-4913-a463-d8e11c0b3c76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:37.939703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:37.939703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:37.939703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:37.939703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:37.939703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:37.939703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:37.939703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:37.939703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:37.939703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:37.939703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:37.939703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:37.939703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:15:37.939703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:15:37.939703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:15:37.940638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:15:37.940638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:37.940638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:37.940638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:37.940638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:37.940638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:37.940638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:37.940638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:37.940638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:37.940638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:37.940638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:37.940638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:37.940638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:37.940638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:37.940638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:37.940638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:37.940638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:37.941516 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:15:37.941516 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:15:37.941516 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:15:37.941516 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:15:37.941516 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:15:37.941516 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:15:37.941516 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:37.941516 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:37.944131 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 381/761] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:37.948873 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc9a55-1f4e-4371-8bc2-a06d0add89a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:37.948873 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc9a55-1f4e-4371-8bc2-a06d0add89a9 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:37.948873 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc9a55-1f4e-4371-8bc2-a06d0add89a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:37.949206 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc9a55-1f4e-4371-8bc2-a06d0add89a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:37.950675 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872e9f2-cde6-4913-a463-d8e11c0b3c76 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:37.969767 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc9a55-1f4e-4371-8bc2-a06d0add89a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:37.971485 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc9a55-1f4e-4371-8bc2-a06d0add89a9 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:37.993539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-55dc9a55-1f4e-4371-8bc2-a06d0add89a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:37.993539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:37.993539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:37.993539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:37.993539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:37.993539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:37.993539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:37.993539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:37.993539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:37.993539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:37.993539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:37.993539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:37.993539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:15:37.993539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:15:37.993539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:15:37.996574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:15:37.996574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:37.996574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:37.996574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:37.996574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:37.996574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:37.996574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:37.996574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:37.996574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:37.996574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:37.996574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:37.996574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:37.996574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:37.996574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:37.996574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:37.996574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:37.996574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:37.997466 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:15:37.997466 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:15:37.997466 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:15:37.997466 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:15:37.997466 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:15:37.997466 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:15:37.997466 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:37.997466 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:37.997466 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 381/762] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:38.002964 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc9a55-1f4e-4371-8bc2-a06d0add89a9 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:38.077074 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872e9f2-cde6-4913-a463-d8e11c0b3c76 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:38.089492 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:38.114191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63718b8e-32db-4023-8438-e7e5e6084c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:38.115031 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63718b8e-32db-4023-8438-e7e5e6084c80 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:38.115416 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63718b8e-32db-4023-8438-e7e5e6084c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:38.115794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63718b8e-32db-4023-8438-e7e5e6084c80 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:38.116797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63718b8e-32db-4023-8438-e7e5e6084c80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:38.117210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63718b8e-32db-4023-8438-e7e5e6084c80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:38.118949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63718b8e-32db-4023-8438-e7e5e6084c80 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:38.131752 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 382/763] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:15:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:38.137536 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63718b8e-32db-4023-8438-e7e5e6084c80 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:38.198359 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc9a55-1f4e-4371-8bc2-a06d0add89a9 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:38.241102 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63718b8e-32db-4023-8438-e7e5e6084c80 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:38.248783 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c273f-0198-4c3b-91d8-c46a0878709a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:38.249863 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c273f-0198-4c3b-91d8-c46a0878709a None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:38.250345 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c273f-0198-4c3b-91d8-c46a0878709a None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:38.250816 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51c273f-0198-4c3b-91d8-c46a0878709a None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:38.252992 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c273f-0198-4c3b-91d8-c46a0878709a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:38.253632 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c273f-0198-4c3b-91d8-c46a0878709a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:38.255351 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51c273f-0198-4c3b-91d8-c46a0878709a None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:38.276721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 382/764] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:15:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:38.282506 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c273f-0198-4c3b-91d8-c46a0878709a None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:38.320638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44157d0-0b38-4469-8eac-4f46d53fbb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:38.321796 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44157d0-0b38-4469-8eac-4f46d53fbb28 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:38.322285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44157d0-0b38-4469-8eac-4f46d53fbb28 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:38.322778 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44157d0-0b38-4469-8eac-4f46d53fbb28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:38.323858 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44157d0-0b38-4469-8eac-4f46d53fbb28 None admin] RBAC: Adding query filter params (role.id=8e315ffd561a48f0891ef53b51d548d8, scope.project.id=6ceffdd955324aec84ba048d2137ae00, user.id=07c9688ca035484ba5341859574c2799) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:38.324383 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44157d0-0b38-4469-8eac-4f46d53fbb28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:38.326404 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44157d0-0b38-4469-8eac-4f46d53fbb28 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:38.338329 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 383/765] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:15:38 2022] GET /identity/v3/role_assignments?user.id=07c9688ca035484ba5341859574c2799&scope.project.id=6ceffdd955324aec84ba048d2137ae00&role.id=8e315ffd561a48f0891ef53b51d548d8 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:15:38.367938 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:38.438189 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf403cc-cb86-4db6-8bf2-2653d76fe8df None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:38.438699 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf403cc-cb86-4db6-8bf2-2653d76fe8df None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:38.438917 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf403cc-cb86-4db6-8bf2-2653d76fe8df None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:38.440504 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 384/766] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:15:38.449100 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd76fc3-79d4-40b6-a12e-9a4bdc3c5b26 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:38.449935 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd76fc3-79d4-40b6-a12e-9a4bdc3c5b26 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:38.450704 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd76fc3-79d4-40b6-a12e-9a4bdc3c5b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:38.477319 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51c273f-0198-4c3b-91d8-c46a0878709a None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:38.485629 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-fdd76fc3-79d4-40b6-a12e-9a4bdc3c5b26 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:38.488218 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-fdd76fc3-79d4-40b6-a12e-9a4bdc3c5b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:38.545885 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:38.557285 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5304f35-5ab8-49f6-a659-774aca47fedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:38.557961 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5304f35-5ab8-49f6-a659-774aca47fedf None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:38.560665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5304f35-5ab8-49f6-a659-774aca47fedf None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:38.560665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5304f35-5ab8-49f6-a659-774aca47fedf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:38.560665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5304f35-5ab8-49f6-a659-774aca47fedf None admin] RBAC: Adding query filter params (role.id=e5a908bd3ed5445b82a6053246533b88, scope.project.id=6ceffdd955324aec84ba048d2137ae00, user.id=21d559cd7127490da84185f20684929d) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:38.560665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5304f35-5ab8-49f6-a659-774aca47fedf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:38.561024 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5304f35-5ab8-49f6-a659-774aca47fedf None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:38.574090 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 383/767] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:15:38 2022] GET /identity/v3/role_assignments?user.id=21d559cd7127490da84185f20684929d&scope.project.id=6ceffdd955324aec84ba048d2137ae00&role.id=e5a908bd3ed5445b82a6053246533b88 => generated 259 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:38.577878 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 385/768] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:15:38.621771 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5304f35-5ab8-49f6-a659-774aca47fedf None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:38.750502 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5304f35-5ab8-49f6-a659-774aca47fedf None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:38.753253 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:38.788471 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0acf2-4231-4023-8a69-9f63606778a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:38.789312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0acf2-4231-4023-8a69-9f63606778a1 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:38.790185 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0acf2-4231-4023-8a69-9f63606778a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:38.790466 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0acf2-4231-4023-8a69-9f63606778a1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:38.791173 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0acf2-4231-4023-8a69-9f63606778a1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:38.792582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0acf2-4231-4023-8a69-9f63606778a1 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:38.806713 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-84a0acf2-4231-4023-8a69-9f63606778a1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:15:38.806713 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:38.806713 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:15:38.806713 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:15:38.806713 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:15:38.806713 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:15:38.806713 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:15:38.806713 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:15:38.806713 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:15:38.806713 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:38.806713 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:15:38.806713 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:38.806713 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:38.806713 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:38.806713 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:38.806713 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:38.807735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:38.807735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:38.807735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:38.807735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:38.807735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:38.807735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:38.807735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:38.807735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:15:38.807735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:15:38.807735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:15:38.807735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:15:38.807735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:38.807735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:38.807735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:38.807735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:38.807735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:38.808572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:38.808572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:38.808572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:38.808572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:38.808572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:38.808572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:38.808572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:38.808572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:38.808572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:38.808572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:38.808572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:38.808572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:15:38.808572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:15:38.808572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:15:38.808572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:38.812423 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 384/769] 10.222.0.24 () {56 vars in 1129 bytes} [Fri May 27 03:15:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:38.814688 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd76fc3-79d4-40b6-a12e-9a4bdc3c5b26 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:38.919541 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd76fc3-79d4-40b6-a12e-9a4bdc3c5b26 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:38.960337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5023aa70-6adb-4fb6-b6f9-2e055bf48a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:38.961020 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023aa70-6adb-4fb6-b6f9-2e055bf48a46 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:38.961301 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023aa70-6adb-4fb6-b6f9-2e055bf48a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:38.961579 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023aa70-6adb-4fb6-b6f9-2e055bf48a46 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:38.962330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5023aa70-6adb-4fb6-b6f9-2e055bf48a46 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:38.962636 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5023aa70-6adb-4fb6-b6f9-2e055bf48a46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:38.965014 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5023aa70-6adb-4fb6-b6f9-2e055bf48a46 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:38.974755 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 386/770] 10.222.0.24 () {56 vars in 1124 bytes} [Fri May 27 03:15:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:38.976248 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0acf2-4231-4023-8a69-9f63606778a1 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:39.088023 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0acf2-4231-4023-8a69-9f63606778a1 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:39.127428 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77368e1-32f2-4352-8af2-d3c49cba2f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:39.128277 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77368e1-32f2-4352-8af2-d3c49cba2f49 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:39.128810 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77368e1-32f2-4352-8af2-d3c49cba2f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:39.129160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77368e1-32f2-4352-8af2-d3c49cba2f49 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:39.129644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236bb48-f98a-4b38-9502-988b8de5a8db None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:39.130005 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236bb48-f98a-4b38-9502-988b8de5a8db None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:39.130127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77368e1-32f2-4352-8af2-d3c49cba2f49 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:39.130426 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236bb48-f98a-4b38-9502-988b8de5a8db None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:39.130535 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77368e1-32f2-4352-8af2-d3c49cba2f49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:39.132011 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77368e1-32f2-4352-8af2-d3c49cba2f49 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:39.132232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 387/771] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:39.137249 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f88ed-01a6-4691-ac4f-0dc4a83a40e6 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:39.137604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f88ed-01a6-4691-ac4f-0dc4a83a40e6 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:39.137957 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f88ed-01a6-4691-ac4f-0dc4a83a40e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:39.142934 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 385/772] 10.222.0.24 () {56 vars in 1124 bytes} [Fri May 27 03:15:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:39.151299 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77368e1-32f2-4352-8af2-d3c49cba2f49 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:39.175707 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e00f88ed-01a6-4691-ac4f-0dc4a83a40e6 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:39.176908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e00f88ed-01a6-4691-ac4f-0dc4a83a40e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:39.223205 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:39.227173 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:39.320578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 388/773] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:15:39.335760 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77368e1-32f2-4352-8af2-d3c49cba2f49 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:39.355720 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f88ed-01a6-4691-ac4f-0dc4a83a40e6 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:39.372108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b08a1-068a-4f28-98be-e0a36cc0fb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:39.372782 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b08a1-068a-4f28-98be-e0a36cc0fb24 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:39.373066 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b08a1-068a-4f28-98be-e0a36cc0fb24 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:39.373344 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b08a1-068a-4f28-98be-e0a36cc0fb24 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:39.374134 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b08a1-068a-4f28-98be-e0a36cc0fb24 None admin] RBAC: Adding query filter params (interface=public, service_id=88206d949ec94a35ad5807dba9bdbcd8, region_id=RegionOne) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:39.374446 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b08a1-068a-4f28-98be-e0a36cc0fb24 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:39.378869 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b08a1-068a-4f28-98be-e0a36cc0fb24 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:39.390399 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 386/774] 10.222.0.24 () {56 vars in 1251 bytes} [Fri May 27 03:15:39 2022] GET /identity/v3/endpoints?service_id=88206d949ec94a35ad5807dba9bdbcd8&interface=public®ion_id=RegionOne => generated 194 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:39.391811 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b08a1-068a-4f28-98be-e0a36cc0fb24 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:39.498663 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b08a1-068a-4f28-98be-e0a36cc0fb24 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:39.519320 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:39.534174 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f12b4-50a6-483c-8425-b782a7f9af18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:39.535048 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f12b4-50a6-483c-8425-b782a7f9af18 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:39.538677 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f12b4-50a6-483c-8425-b782a7f9af18 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:39.538677 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f12b4-50a6-483c-8425-b782a7f9af18 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:39.538677 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7f12b4-50a6-483c-8425-b782a7f9af18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:39.538677 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7f12b4-50a6-483c-8425-b782a7f9af18 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:39.548681 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 387/775] 10.222.0.24 () {56 vars in 1087 bytes} [Fri May 27 03:15:39 2022] GET /identity/v3/services => generated 772 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:39.616931 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f88ed-01a6-4691-ac4f-0dc4a83a40e6 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:39.673571 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d80ef6-3706-4b90-8199-bf2b9bffae5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:39.674378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d80ef6-3706-4b90-8199-bf2b9bffae5e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:39.674661 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d80ef6-3706-4b90-8199-bf2b9bffae5e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:39.674966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d80ef6-3706-4b90-8199-bf2b9bffae5e None admin] PATH_INFO: `/v3/services/network` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:39.675631 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d80ef6-3706-4b90-8199-bf2b9bffae5e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:39.676998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d80ef6-3706-4b90-8199-bf2b9bffae5e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:39.698459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-28d80ef6-3706-4b90-8199-bf2b9bffae5e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:15:39.698459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:39.698459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:15:39.698459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:15:39.698459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:15:39.698459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:15:39.698459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:15:39.698459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:15:39.698459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:15:39.698459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:39.698459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:15:39.698459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:39.698459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:39.698459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:39.698459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:39.698459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:39.698459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:39.699521 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:39.699521 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:39.699521 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:39.699521 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:39.699521 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:39.699521 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:39.699521 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:15:39.699521 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:15:39.699521 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:15:39.699521 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:15:39.699521 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:39.699521 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:39.699521 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:39.699521 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:39.699521 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:39.699521 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:39.700609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:39.700609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:39.700609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:39.700609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:39.700609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:39.700609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:39.700609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:39.700609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:39.700609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:39.700609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:39.700609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:15:39.700609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:15:39.700609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:15:39.700609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:39.704188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 389/776] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:39 2022] GET /identity/v3/services/network => generated 88 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:39.710075 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f12b4-50a6-483c-8425-b782a7f9af18 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:39.820660 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f12b4-50a6-483c-8425-b782a7f9af18 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:39.857484 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ff93b-5309-474e-ba4d-de19b26bb1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:39.858156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ff93b-5309-474e-ba4d-de19b26bb1d7 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:39.858439 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ff93b-5309-474e-ba4d-de19b26bb1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:39.858715 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ff93b-5309-474e-ba4d-de19b26bb1d7 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:39.859452 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ff93b-5309-474e-ba4d-de19b26bb1d7 None admin] RBAC: Adding query filter params (name=network) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:39.859759 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ff93b-5309-474e-ba4d-de19b26bb1d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:39.861122 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ff93b-5309-474e-ba4d-de19b26bb1d7 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:39.866232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 388/777] 10.222.0.24 () {56 vars in 1112 bytes} [Fri May 27 03:15:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:39.872204 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d80ef6-3706-4b90-8199-bf2b9bffae5e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:39.903242 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:39.976562 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d80ef6-3706-4b90-8199-bf2b9bffae5e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:40.012692 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de43dbf0-d5cd-4ebc-a25c-87f4b7d4faeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:40.013361 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43dbf0-d5cd-4ebc-a25c-87f4b7d4faeb None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:40.013645 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43dbf0-d5cd-4ebc-a25c-87f4b7d4faeb None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:40.013918 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de43dbf0-d5cd-4ebc-a25c-87f4b7d4faeb None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:40.014636 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de43dbf0-d5cd-4ebc-a25c-87f4b7d4faeb None admin] RBAC: Adding query filter params (type=network) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:40.014968 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de43dbf0-d5cd-4ebc-a25c-87f4b7d4faeb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:40.016348 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de43dbf0-d5cd-4ebc-a25c-87f4b7d4faeb None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:40.021757 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 390/778] 10.222.0.24 () {56 vars in 1112 bytes} [Fri May 27 03:15:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:40.027548 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ff93b-5309-474e-ba4d-de19b26bb1d7 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:40.074392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:40.180409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ff93b-5309-474e-ba4d-de19b26bb1d7 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:40.231572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ee20a-863b-4c61-8abe-90b94eef113e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:40.232615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ee20a-863b-4c61-8abe-90b94eef113e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:40.233101 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ee20a-863b-4c61-8abe-90b94eef113e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:40.233574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ee20a-863b-4c61-8abe-90b94eef113e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:40.234573 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ee20a-863b-4c61-8abe-90b94eef113e None admin] RBAC: Adding query filter params (interface=public, service_id=5cf481d7fb534f4baa7d7d90d4b3d4cf, region_id=RegionOne) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:40.235108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ee20a-863b-4c61-8abe-90b94eef113e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:40.236691 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ee20a-863b-4c61-8abe-90b94eef113e None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:40.247819 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 389/779] 10.222.0.24 () {56 vars in 1251 bytes} [Fri May 27 03:15:40 2022] GET /identity/v3/endpoints?service_id=5cf481d7fb534f4baa7d7d90d4b3d4cf&interface=public®ion_id=RegionOne => generated 194 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:15:40.252883 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de43dbf0-d5cd-4ebc-a25c-87f4b7d4faeb None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:40.435968 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:40.473064 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de43dbf0-d5cd-4ebc-a25c-87f4b7d4faeb None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:40.528478 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79dcbe-919c-4468-ba19-852b65b29f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:40.529140 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79dcbe-919c-4468-ba19-852b65b29f07 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:40.529416 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79dcbe-919c-4468-ba19-852b65b29f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:40.529687 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79dcbe-919c-4468-ba19-852b65b29f07 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:40.530369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79dcbe-919c-4468-ba19-852b65b29f07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:40.531752 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79dcbe-919c-4468-ba19-852b65b29f07 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:40.539415 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 391/780] 10.222.0.24 () {56 vars in 1087 bytes} [Fri May 27 03:15:40 2022] GET /identity/v3/services => generated 772 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:40.732618 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b923d-dc9f-4e61-aac0-d0ba70426ded None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:40.733122 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b923d-dc9f-4e61-aac0-d0ba70426ded None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:40.733122 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b923d-dc9f-4e61-aac0-d0ba70426ded None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:40.734664 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 390/781] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:40.740588 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fad666-8160-4649-85eb-eca7d4e55b93 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:40.741098 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fad666-8160-4649-85eb-eca7d4e55b93 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:40.741175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fad666-8160-4649-85eb-eca7d4e55b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:40.768074 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-b6fad666-8160-4649-85eb-eca7d4e55b93 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:40.769311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b6fad666-8160-4649-85eb-eca7d4e55b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:40.846057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 392/782] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:15:40.870332 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b923d-dc9f-4e61-aac0-d0ba70426ded None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:40.963999 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:40.976574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b923d-dc9f-4e61-aac0-d0ba70426ded None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:41.014476 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d690f365-887d-42e3-b4bc-9016d9789a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:41.015268 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690f365-887d-42e3-b4bc-9016d9789a78 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:41.015543 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690f365-887d-42e3-b4bc-9016d9789a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:41.015810 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690f365-887d-42e3-b4bc-9016d9789a78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:41.026946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d690f365-887d-42e3-b4bc-9016d9789a78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:41.034725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-d690f365-887d-42e3-b4bc-9016d9789a78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:15:41.034725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:41.034725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:41.034725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:41.034725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:41.034725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:41.034725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:41.034725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:41.034725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:41.034725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:41.034725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:41.034725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:41.034725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:41.034725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:41.034725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:41.034725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:41.035686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:41.035686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:41.035686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:41.035686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:41.035686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:41.035686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:41.035686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:41.035686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:41.035686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:41.035686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:41.035686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:41.035686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:41.035686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:41.035686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:41.035686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:41.035686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:41.036655 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:41.036655 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:41.036655 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:41.036655 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:41.036655 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:41.036655 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:41.036655 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:41.036655 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:41.036655 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:41.036655 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:41.036655 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:15:41.036655 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:41.036655 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 391/783] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:15:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:41.042379 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fad666-8160-4649-85eb-eca7d4e55b93 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:41.058623 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755e8bf-22ce-43d4-b5be-d1ae1a9414ae None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:41.058623 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755e8bf-22ce-43d4-b5be-d1ae1a9414ae None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:41.058623 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755e8bf-22ce-43d4-b5be-d1ae1a9414ae None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:41.058623 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 392/784] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:15:41.068323 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dae208-6ff1-46d5-905a-2b2e9a93b78f None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:41.068828 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dae208-6ff1-46d5-905a-2b2e9a93b78f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:41.069306 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dae208-6ff1-46d5-905a-2b2e9a93b78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:41.088115 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-46dae208-6ff1-46d5-905a-2b2e9a93b78f None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:41.090305 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-46dae208-6ff1-46d5-905a-2b2e9a93b78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:41.183600 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 393/785] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:15:41.203840 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dae208-6ff1-46d5-905a-2b2e9a93b78f None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:41.212793 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:41.243635 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fad666-8160-4649-85eb-eca7d4e55b93 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:41.291227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:41.313188 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dae208-6ff1-46d5-905a-2b2e9a93b78f None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:41.349134 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953bf06f-7ad7-4ea0-a812-a6afe8dea74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:41.349822 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bf06f-7ad7-4ea0-a812-a6afe8dea74e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:41.350107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bf06f-7ad7-4ea0-a812-a6afe8dea74e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:41.350385 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953bf06f-7ad7-4ea0-a812-a6afe8dea74e None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:41.351276 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953bf06f-7ad7-4ea0-a812-a6afe8dea74e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:41.351590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953bf06f-7ad7-4ea0-a812-a6afe8dea74e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:41.351762 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8eec4-63a4-489e-a6de-20c4c51f3afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:41.353287 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8eec4-63a4-489e-a6de-20c4c51f3afb None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:41.353758 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8eec4-63a4-489e-a6de-20c4c51f3afb None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:41.353862 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953bf06f-7ad7-4ea0-a812-a6afe8dea74e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:41.354293 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8eec4-63a4-489e-a6de-20c4c51f3afb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:41.366337 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8eec4-63a4-489e-a6de-20c4c51f3afb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:41.374686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-27f8eec4-63a4-489e-a6de-20c4c51f3afb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:41.374686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:41.374686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:41.374686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:41.374686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:41.374686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:41.374686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:41.374686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:41.374686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:41.374686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:41.374686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:41.374686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:41.374686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:41.374686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:41.374686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:41.374686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:41.375626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:41.375626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:41.375626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:41.375626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:41.375626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:41.375626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:41.375626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:41.375626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:41.375626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:41.375626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:41.375626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:41.375626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:41.375626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:41.375626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:41.375626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:41.375626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:41.376457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:41.376457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:41.376457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:41.376457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:41.376457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:41.376457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:41.376457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:41.376457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:41.376457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:41.376457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:41.376457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:41.376457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:41.381662 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 393/787] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:15:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:15:41.381876 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 394/787] 10.222.0.24 () {56 vars in 1102 bytes} [Fri May 27 03:15:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:41.385677 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953bf06f-7ad7-4ea0-a812-a6afe8dea74e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:41.386662 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8eec4-63a4-489e-a6de-20c4c51f3afb None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:41.512653 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953bf06f-7ad7-4ea0-a812-a6afe8dea74e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:41.526124 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8eec4-63a4-489e-a6de-20c4c51f3afb None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:41.549329 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6142853-e0ef-4366-828b-844037c8f1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:41.550078 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6142853-e0ef-4366-828b-844037c8f1a4 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:41.550465 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6142853-e0ef-4366-828b-844037c8f1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:41.550854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6142853-e0ef-4366-828b-844037c8f1a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:41.551724 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6142853-e0ef-4366-828b-844037c8f1a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:41.552138 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6142853-e0ef-4366-828b-844037c8f1a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:41.553629 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6142853-e0ef-4366-828b-844037c8f1a4 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:41.566711 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 394/788] 10.222.0.24 () {56 vars in 1103 bytes} [Fri May 27 03:15:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:15:41.566711 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20ad84-1cae-446b-8099-39d870ad738b None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:41.566711 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20ad84-1cae-446b-8099-39d870ad738b None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:41.566711 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20ad84-1cae-446b-8099-39d870ad738b None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:41.569648 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 395/789] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:41.570431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20ad84-1cae-446b-8099-39d870ad738b None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:41.575620 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:41.577300 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b2fd98-8ffa-4ac2-8d5e-0d52333182f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:41.577435 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2fd98-8ffa-4ac2-8d5e-0d52333182f4 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:41.577687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2fd98-8ffa-4ac2-8d5e-0d52333182f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:41.577960 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2fd98-8ffa-4ac2-8d5e-0d52333182f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:41.587188 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:41.595958 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b2fd98-8ffa-4ac2-8d5e-0d52333182f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:41.595958 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b2fd98-8ffa-4ac2-8d5e-0d52333182f4 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:41.610213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-b8b2fd98-8ffa-4ac2-8d5e-0d52333182f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:41.610213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:41.610213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:41.610213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:41.610213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:41.610213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:41.610213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:41.610213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:41.610213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:41.610213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:41.610213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:41.610213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:41.610213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:41.610213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:41.610213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:41.611153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:41.611153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:41.611153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:41.611153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:41.611153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:41.611153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:41.611153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:41.611153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:41.611153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:41.611153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:41.611153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:41.611153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:41.611153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:41.611153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:41.611153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:41.611153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:41.611153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:41.612203 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:41.612203 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:41.612203 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:41.612203 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:41.612203 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 395/790] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:41.613763 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1865b03-bb76-47f1-9a62-4d9c3bc545a5 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:41.614121 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1865b03-bb76-47f1-9a62-4d9c3bc545a5 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:41.614492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1865b03-bb76-47f1-9a62-4d9c3bc545a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:41.632170 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-b1865b03-bb76-47f1-9a62-4d9c3bc545a5 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:41.634063 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b1865b03-bb76-47f1-9a62-4d9c3bc545a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:41.699875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20ad84-1cae-446b-8099-39d870ad738b None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:41.720478 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 396/791] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:15:41.721602 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1865b03-bb76-47f1-9a62-4d9c3bc545a5 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:41.758670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0c510-d87c-4c67-bd27-81398e1fc78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:41.759795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0c510-d87c-4c67-bd27-81398e1fc78e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:41.760176 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0c510-d87c-4c67-bd27-81398e1fc78e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:41.760553 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0c510-d87c-4c67-bd27-81398e1fc78e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:41.772864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e0c510-d87c-4c67-bd27-81398e1fc78e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:41.775214 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e0c510-d87c-4c67-bd27-81398e1fc78e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:41.795431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-76e0c510-d87c-4c67-bd27-81398e1fc78e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:41.795431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:41.795431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:41.795431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:41.795431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:41.795431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:41.795431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:41.795431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:41.795431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:41.795431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:41.795431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:41.795431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:41.795431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:41.795431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:41.795431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:41.796570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:41.796570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:41.796570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:41.796570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:41.796570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:41.796570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:41.796570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:41.796570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:41.796570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:41.796570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:41.796570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:41.796570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:41.796570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:41.796570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:41.796570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:41.796570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:41.796570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:41.797627 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:41.797627 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:41.797627 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:41.797627 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:41.803281 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 396/792] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:41.804163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0c510-d87c-4c67-bd27-81398e1fc78e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:41.846814 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1865b03-bb76-47f1-9a62-4d9c3bc545a5 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:41.863546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:41.871209 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:41.885385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce34fcb-341f-481c-9212-c5583b436260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:41.886358 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce34fcb-341f-481c-9212-c5583b436260 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:41.886818 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce34fcb-341f-481c-9212-c5583b436260 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:41.887322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce34fcb-341f-481c-9212-c5583b436260 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:41.888220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce34fcb-341f-481c-9212-c5583b436260 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:41.888711 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce34fcb-341f-481c-9212-c5583b436260 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:41.890282 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce34fcb-341f-481c-9212-c5583b436260 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:41.904265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 397/793] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:41.905558 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce34fcb-341f-481c-9212-c5583b436260 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:41.910914 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0c510-d87c-4c67-bd27-81398e1fc78e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:41.947228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b983d3-b71a-414c-bb90-96ea0f876059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:41.948056 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b983d3-b71a-414c-bb90-96ea0f876059 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:41.948337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b983d3-b71a-414c-bb90-96ea0f876059 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:41.948613 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b983d3-b71a-414c-bb90-96ea0f876059 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:41.960243 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b983d3-b71a-414c-bb90-96ea0f876059 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:41.968875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-f9b983d3-b71a-414c-bb90-96ea0f876059 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:15:41.968875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:41.968875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:41.968875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:41.968875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:41.968875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:41.968875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:41.968875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:41.968875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:41.968875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:41.968875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:41.968875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:41.968875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:41.968875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:41.968875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:41.968875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:41.969838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:41.969838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:41.969838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:41.969838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:41.969838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:41.969838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:41.969838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:41.969838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:41.969838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:41.969838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:41.969838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:41.969838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:41.969838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:41.969838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:41.969838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:41.969838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:41.970667 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:41.970667 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:41.970667 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:41.970667 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:41.970667 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:41.970667 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:41.970667 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:41.970667 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:41.970667 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:41.970667 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:41.970667 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:15:41.970667 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:41.975147 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 397/794] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:15:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:41.976094 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b983d3-b71a-414c-bb90-96ea0f876059 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:42.018644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce34fcb-341f-481c-9212-c5583b436260 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:42.058019 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a376454-1125-4b55-be53-d455a87a2339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:42.059244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a376454-1125-4b55-be53-d455a87a2339 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:42.059757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a376454-1125-4b55-be53-d455a87a2339 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:42.060256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a376454-1125-4b55-be53-d455a87a2339 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:42.061323 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a376454-1125-4b55-be53-d455a87a2339 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:42.061851 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a376454-1125-4b55-be53-d455a87a2339 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:42.063536 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a376454-1125-4b55-be53-d455a87a2339 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:42.079120 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 398/795] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:42.080447 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a376454-1125-4b55-be53-d455a87a2339 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:42.091739 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b983d3-b71a-414c-bb90-96ea0f876059 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:42.109432 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:42.128527 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8b3e9-1470-4827-b3ec-0f8709bac2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:42.129324 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8b3e9-1470-4827-b3ec-0f8709bac2dc None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:42.129604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8b3e9-1470-4827-b3ec-0f8709bac2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:42.130165 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8b3e9-1470-4827-b3ec-0f8709bac2dc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:42.138534 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8b3e9-1470-4827-b3ec-0f8709bac2dc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:42.139202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:42.140400 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8b3e9-1470-4827-b3ec-0f8709bac2dc None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:42.155178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-55b8b3e9-1470-4827-b3ec-0f8709bac2dc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:15:42.155178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:42.155178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:42.155178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:42.155178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:42.155178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:42.155178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:42.155178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:42.155178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:42.155178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:42.155178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:42.155178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:42.155178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:15:42.155178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:15:42.155178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:15:42.155178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:15:42.156156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:42.156156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:42.156156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:15:42.156156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:42.156156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:15:42.156156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:42.156156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:42.156156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:42.156156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:42.156156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:42.156156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:42.156156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:42.156156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:42.156156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:42.156156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:42.156156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:42.156156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:42.156156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:42.157708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:42.157708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:42.157708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:15:42.157708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:15:42.157708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:15:42.157708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:15:42.157708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:15:42.157708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:15:42.157708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:15:42.157708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:42.157708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 398/796] 10.222.0.24 () {56 vars in 1093 bytes} [Fri May 27 03:15:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:42.157708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8b3e9-1470-4827-b3ec-0f8709bac2dc None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:42.197468 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a376454-1125-4b55-be53-d455a87a2339 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:42.234238 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ab3fa-c7b6-4782-bfed-c7786150e060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:42.235296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ab3fa-c7b6-4782-bfed-c7786150e060 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:42.235780 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ab3fa-c7b6-4782-bfed-c7786150e060 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:42.236256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ab3fa-c7b6-4782-bfed-c7786150e060 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:42.237320 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ab3fa-c7b6-4782-bfed-c7786150e060 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:42.237967 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ab3fa-c7b6-4782-bfed-c7786150e060 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:42.239656 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628ab3fa-c7b6-4782-bfed-c7786150e060 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:42.249124 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 399/797] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:15:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:42.250162 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ab3fa-c7b6-4782-bfed-c7786150e060 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:42.267720 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8b3e9-1470-4827-b3ec-0f8709bac2dc None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:42.303699 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f8d6cf-3751-40e3-820a-08587221165f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:42.304514 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8d6cf-3751-40e3-820a-08587221165f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:42.304798 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8d6cf-3751-40e3-820a-08587221165f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:42.305455 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f8d6cf-3751-40e3-820a-08587221165f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:42.313686 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f8d6cf-3751-40e3-820a-08587221165f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:42.315110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f8d6cf-3751-40e3-820a-08587221165f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:42.329856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-54f8d6cf-3751-40e3-820a-08587221165f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:15:42.329856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:42.329856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:42.329856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:42.329856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:42.329856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:42.329856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:42.329856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:42.329856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:42.329856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:42.329856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:42.329856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:42.329856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:15:42.329856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:15:42.329856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:15:42.329856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:15:42.330789 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:42.330789 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:42.330789 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:15:42.330789 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:42.330789 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:15:42.330789 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:42.330789 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:42.330789 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:42.330789 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:42.330789 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:42.330789 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:42.330789 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:42.330789 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:42.330789 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:42.330789 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:42.330789 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:42.330789 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:42.330789 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:42.331746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:42.331746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:42.331746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:15:42.331746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:15:42.331746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:15:42.331746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:15:42.331746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:15:42.331746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:15:42.331746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:15:42.331746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:42.331746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 399/798] 10.222.0.24 () {56 vars in 1117 bytes} [Fri May 27 03:15:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:42.332350 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f8d6cf-3751-40e3-820a-08587221165f None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:42.359875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ab3fa-c7b6-4782-bfed-c7786150e060 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:42.361973 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:42.375218 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:42.395372 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b002f13-e1f5-470b-b737-54ad761d5daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:42.396039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b002f13-e1f5-470b-b737-54ad761d5daa None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:42.396343 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b002f13-e1f5-470b-b737-54ad761d5daa None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:42.396593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b002f13-e1f5-470b-b737-54ad761d5daa None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:42.397429 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b002f13-e1f5-470b-b737-54ad761d5daa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:42.397737 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b002f13-e1f5-470b-b737-54ad761d5daa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:42.399365 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b002f13-e1f5-470b-b737-54ad761d5daa None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:42.417409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 400/799] 10.222.0.24 () {56 vars in 1133 bytes} [Fri May 27 03:15:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:42.418908 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbef5b-380a-4304-b97c-d8c3e6a322ac None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:42.419381 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbef5b-380a-4304-b97c-d8c3e6a322ac None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:42.419656 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbef5b-380a-4304-b97c-d8c3e6a322ac None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:42.421068 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 401/800] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:42.421602 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fbef5b-380a-4304-b97c-d8c3e6a322ac None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:42.460116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f8d6cf-3751-40e3-820a-08587221165f None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:42.498115 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a79a113-53b1-46d6-8625-f2c9b58d308b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:42.499171 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79a113-53b1-46d6-8625-f2c9b58d308b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:42.499597 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79a113-53b1-46d6-8625-f2c9b58d308b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:42.500041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79a113-53b1-46d6-8625-f2c9b58d308b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:42.511162 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a79a113-53b1-46d6-8625-f2c9b58d308b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:42.513043 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a79a113-53b1-46d6-8625-f2c9b58d308b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:42.535495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-0a79a113-53b1-46d6-8625-f2c9b58d308b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:42.535495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:42.535495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:42.535495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:42.535495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:42.535495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:42.535495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:42.535495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:42.535495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:42.535495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:42.535495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:42.535495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:42.535495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:42.535495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:42.535495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:42.536434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:42.536434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:42.536434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:42.536434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:42.536434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:42.536434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:42.536434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:42.536434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:42.536434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:42.536434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:42.536434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:42.536434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:42.536434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:42.536434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:42.536434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:42.536434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:42.536434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:42.537308 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:42.537308 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:42.537308 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:42.537308 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:42.537308 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 400/801] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:42.538095 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a79a113-53b1-46d6-8625-f2c9b58d308b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:42.546117 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fbef5b-380a-4304-b97c-d8c3e6a322ac None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:42.583146 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0995f03-dca9-4658-b98c-74fe5814b547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:42.583732 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0995f03-dca9-4658-b98c-74fe5814b547 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:42.584328 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0995f03-dca9-4658-b98c-74fe5814b547 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:42.584608 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0995f03-dca9-4658-b98c-74fe5814b547 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:42.585597 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0995f03-dca9-4658-b98c-74fe5814b547 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:42.585904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0995f03-dca9-4658-b98c-74fe5814b547 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:42.587371 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0995f03-dca9-4658-b98c-74fe5814b547 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:42.604803 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 402/802] 10.222.0.24 () {56 vars in 1149 bytes} [Fri May 27 03:15:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:42.606049 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4ead0-8a97-466e-af13-d6b4bdc69b7f None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:42.606331 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4ead0-8a97-466e-af13-d6b4bdc69b7f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:42.606599 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4ead0-8a97-466e-af13-d6b4bdc69b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:42.624713 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-f9e4ead0-8a97-466e-af13-d6b4bdc69b7f None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:42.626108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f9e4ead0-8a97-466e-af13-d6b4bdc69b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:42.632470 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:42.642461 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:42.645275 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a79a113-53b1-46d6-8625-f2c9b58d308b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:42.678797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04fc637-bb32-47fa-bd88-51b1d3f81bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:42.679522 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04fc637-bb32-47fa-bd88-51b1d3f81bef None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:42.679804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04fc637-bb32-47fa-bd88-51b1d3f81bef None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:42.680647 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04fc637-bb32-47fa-bd88-51b1d3f81bef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:42.696076 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 403/803] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:15:42.696828 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4ead0-8a97-466e-af13-d6b4bdc69b7f None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:42.762992 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04fc637-bb32-47fa-bd88-51b1d3f81bef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:42.764461 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04fc637-bb32-47fa-bd88-51b1d3f81bef None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:42.804067 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4ead0-8a97-466e-af13-d6b4bdc69b7f None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:42.842328 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f4812-8162-4780-80aa-75865557e65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:42.842944 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f4812-8162-4780-80aa-75865557e65b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:42.843223 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f4812-8162-4780-80aa-75865557e65b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:42.843502 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f4812-8162-4780-80aa-75865557e65b None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:42.844220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f4812-8162-4780-80aa-75865557e65b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:42.844547 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f4812-8162-4780-80aa-75865557e65b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:42.845910 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f4812-8162-4780-80aa-75865557e65b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:42.862204 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 404/804] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:42.862204 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f4812-8162-4780-80aa-75865557e65b None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:42.867866 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:42.870348 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:42.886401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 401/805] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:15:42 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:15:42.886401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04fc637-bb32-47fa-bd88-51b1d3f81bef None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:42.967972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f4812-8162-4780-80aa-75865557e65b None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:42.991912 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04fc637-bb32-47fa-bd88-51b1d3f81bef None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:43.009738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82bdae-09d3-46a3-9209-e54dded51fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:43.011014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82bdae-09d3-46a3-9209-e54dded51fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:43.011295 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82bdae-09d3-46a3-9209-e54dded51fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:43.011570 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82bdae-09d3-46a3-9209-e54dded51fc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:43.022949 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d82bdae-09d3-46a3-9209-e54dded51fc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:43.025214 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d82bdae-09d3-46a3-9209-e54dded51fc2 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:43.028375 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11182da-daa5-4940-9258-44af1cb090a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:43.029185 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11182da-daa5-4940-9258-44af1cb090a3 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:43.029462 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11182da-daa5-4940-9258-44af1cb090a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:43.029731 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11182da-daa5-4940-9258-44af1cb090a3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:43.030381 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11182da-daa5-4940-9258-44af1cb090a3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:43.031766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11182da-daa5-4940-9258-44af1cb090a3 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:43.042830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-9d82bdae-09d3-46a3-9209-e54dded51fc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:43.042830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:43.042830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:43.042830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:43.042830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:43.042830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:43.042830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:43.042830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:43.042830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:43.042830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:43.042830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:43.042830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:43.042830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:43.042830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:43.042830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:43.043716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:43.043716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:43.043716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:43.043716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:43.043716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:43.043716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:43.043716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:43.043716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:43.043716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:43.043716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:43.043716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:43.043716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:43.043716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:43.043716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:43.043716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:43.043716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:43.043716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:43.045268 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:43.045268 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:43.045268 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:43.045268 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:43.045491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-b11182da-daa5-4940-9258-44af1cb090a3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:15:43.045491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:43.045491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:15:43.045491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:15:43.045491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:15:43.045491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:15:43.045491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:15:43.045491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:15:43.045491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:15:43.045491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:43.045491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:15:43.045491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:43.045491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:43.045491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:43.045491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:43.045491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:43.046336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:43.046336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:43.046336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:43.046336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:43.046336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:43.046336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:43.046336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:43.046336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:15:43.046336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:15:43.046336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:15:43.046336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:15:43.046336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:43.046336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:43.046336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:43.046336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:43.046336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:43.047180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:43.047180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:43.047180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:43.047180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:43.047180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:43.047180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:43.047180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:43.047180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:43.047180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:43.047180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:43.047180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:43.047180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:15:43.047180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:15:43.047180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:15:43.047180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:43.047959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 405/806] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:43.047959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82bdae-09d3-46a3-9209-e54dded51fc2 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:43.049215 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 402/807] 10.222.0.24 () {56 vars in 1129 bytes} [Fri May 27 03:15:42 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:43.050067 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11182da-daa5-4940-9258-44af1cb090a3 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:43.116827 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:43.120323 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:43.164401 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82bdae-09d3-46a3-9209-e54dded51fc2 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:43.170109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11182da-daa5-4940-9258-44af1cb090a3 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:43.198776 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09356bd8-3b5a-4249-9ecc-508c252f9858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:43.199665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09356bd8-3b5a-4249-9ecc-508c252f9858 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:43.199949 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09356bd8-3b5a-4249-9ecc-508c252f9858 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:43.200463 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09356bd8-3b5a-4249-9ecc-508c252f9858 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:43.203551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e9ed7-6b03-4834-a861-4066c2dd69e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:43.204330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e9ed7-6b03-4834-a861-4066c2dd69e4 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:43.204613 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e9ed7-6b03-4834-a861-4066c2dd69e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:43.204890 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1e9ed7-6b03-4834-a861-4066c2dd69e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:43.208614 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09356bd8-3b5a-4249-9ecc-508c252f9858 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:43.210036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09356bd8-3b5a-4249-9ecc-508c252f9858 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:43.215731 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e9ed7-6b03-4834-a861-4066c2dd69e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:43.217155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1e9ed7-6b03-4834-a861-4066c2dd69e4 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:43.225039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-09356bd8-3b5a-4249-9ecc-508c252f9858 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 03:15:43.225039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:43.225039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:43.225039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:43.225039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:43.225039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:43.225039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:43.225039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:43.225039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:43.225039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:43.225039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:43.225039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:43.225039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:15:43.225039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:15:43.225039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:15:43.225039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:15:43.226002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:43.226002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:43.226002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:15:43.226002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:43.226002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:15:43.226002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:43.226002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:43.226002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:43.226002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:43.226002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:43.226002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:43.226002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:43.226002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:43.226002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:43.226002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:43.226002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:43.226002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:43.226002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:43.227112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:43.227112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:43.227112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:15:43.227112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:15:43.227112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:15:43.227112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:15:43.227112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:15:43.227112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:15:43.227112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 03:15:43.227112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:43.227112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 406/808] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:15:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:43.227112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09356bd8-3b5a-4249-9ecc-508c252f9858 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:43.234089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-cb1e9ed7-6b03-4834-a861-4066c2dd69e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:43.234089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:43.234089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:43.234089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:43.234089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:43.234089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:43.234089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:43.234089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:43.234089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:43.234089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:43.234089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:43.234089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:43.234089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:43.234089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:43.234089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:43.234998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:43.234998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:43.234998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:43.234998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:43.234998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:43.234998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:43.234998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:43.234998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:43.234998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:43.234998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:43.234998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:43.234998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:43.234998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:43.234998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:43.234998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:43.234998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:43.234998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:43.235908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:43.235908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:43.235908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:43.235908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:43.235908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 403/809] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:43.236192 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e9ed7-6b03-4834-a861-4066c2dd69e4 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:43.327280 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09356bd8-3b5a-4249-9ecc-508c252f9858 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:43.334900 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1e9ed7-6b03-4834-a861-4066c2dd69e4 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:43.337129 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:43.347480 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:43.360993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55bc442-7f3e-4957-ac15-05e54f99ec77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:43.361650 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55bc442-7f3e-4957-ac15-05e54f99ec77 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:43.361934 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55bc442-7f3e-4957-ac15-05e54f99ec77 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:43.362216 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55bc442-7f3e-4957-ac15-05e54f99ec77 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:43.362995 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55bc442-7f3e-4957-ac15-05e54f99ec77 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:43.363304 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55bc442-7f3e-4957-ac15-05e54f99ec77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:43.364660 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55bc442-7f3e-4957-ac15-05e54f99ec77 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:43.366911 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ed01f-17c4-41ef-a290-f259bddf2355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:43.367517 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ed01f-17c4-41ef-a290-f259bddf2355 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:43.368293 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ed01f-17c4-41ef-a290-f259bddf2355 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:43.368565 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ed01f-17c4-41ef-a290-f259bddf2355 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:43.369305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ed01f-17c4-41ef-a290-f259bddf2355 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:43.374401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ed01f-17c4-41ef-a290-f259bddf2355 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:43.374558 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 407/810] 10.222.0.24 () {56 vars in 1124 bytes} [Fri May 27 03:15:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:43.374558 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e69de-0d40-4e56-8900-cded24dd7a46 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:43.374558 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e69de-0d40-4e56-8900-cded24dd7a46 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:43.374790 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ed01f-17c4-41ef-a290-f259bddf2355 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:43.374907 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8e69de-0d40-4e56-8900-cded24dd7a46 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:43.374907 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 408/811] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:43.374907 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e69de-0d40-4e56-8900-cded24dd7a46 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:43.391997 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 404/812] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:43.391997 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ed01f-17c4-41ef-a290-f259bddf2355 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:43.485534 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8e69de-0d40-4e56-8900-cded24dd7a46 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:43.498721 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ed01f-17c4-41ef-a290-f259bddf2355 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:43.518553 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3f704-ac3f-46c3-9765-4b9be4492f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:43.519248 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3f704-ac3f-46c3-9765-4b9be4492f22 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:43.519532 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3f704-ac3f-46c3-9765-4b9be4492f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:43.519808 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3f704-ac3f-46c3-9765-4b9be4492f22 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:43.520667 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3f704-ac3f-46c3-9765-4b9be4492f22 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:43.520986 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3f704-ac3f-46c3-9765-4b9be4492f22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:43.522410 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3f704-ac3f-46c3-9765-4b9be4492f22 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:43.531061 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b424eed0-ede0-498d-960a-60c4b56a27fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:43.531719 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424eed0-ede0-498d-960a-60c4b56a27fd None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:43.532003 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424eed0-ede0-498d-960a-60c4b56a27fd None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:43.532265 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424eed0-ede0-498d-960a-60c4b56a27fd None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:43.532979 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b424eed0-ede0-498d-960a-60c4b56a27fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:43.533282 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b424eed0-ede0-498d-960a-60c4b56a27fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:43.534626 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b424eed0-ede0-498d-960a-60c4b56a27fd None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:43.539237 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 409/813] 10.222.0.24 () {56 vars in 1135 bytes} [Fri May 27 03:15:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:43.540071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3f704-ac3f-46c3-9765-4b9be4492f22 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:43.551071 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 405/814] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:43.551071 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76590309-3946-4f18-932b-9c400281bb29 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:43.551071 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76590309-3946-4f18-932b-9c400281bb29 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:43.551071 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76590309-3946-4f18-932b-9c400281bb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:43.564489 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-76590309-3946-4f18-932b-9c400281bb29 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:43.565663 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-76590309-3946-4f18-932b-9c400281bb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:43.582912 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:43.587109 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:43.629984 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 406/815] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:15:43.630732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76590309-3946-4f18-932b-9c400281bb29 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:43.654748 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3f704-ac3f-46c3-9765-4b9be4492f22 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:43.688895 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff656de-60f6-4cad-9f19-ffd3511062ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:43.689553 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff656de-60f6-4cad-9f19-ffd3511062ab None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:43.689835 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff656de-60f6-4cad-9f19-ffd3511062ab None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:43.690115 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff656de-60f6-4cad-9f19-ffd3511062ab None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:43.690812 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff656de-60f6-4cad-9f19-ffd3511062ab None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:43.691151 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff656de-60f6-4cad-9f19-ffd3511062ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:43.692505 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff656de-60f6-4cad-9f19-ffd3511062ab None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:43.697253 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 410/816] 10.222.0.24 () {56 vars in 1124 bytes} [Fri May 27 03:15:43 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:43.698019 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff656de-60f6-4cad-9f19-ffd3511062ab None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:43.728818 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76590309-3946-4f18-932b-9c400281bb29 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:43.761504 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d280047-42fc-436b-bbb5-16b1f6d992e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:43.762286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d280047-42fc-436b-bbb5-16b1f6d992e8 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:43.762568 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d280047-42fc-436b-bbb5-16b1f6d992e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:43.762868 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d280047-42fc-436b-bbb5-16b1f6d992e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:43.769355 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d280047-42fc-436b-bbb5-16b1f6d992e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:43.770796 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d280047-42fc-436b-bbb5-16b1f6d992e8 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:43.784938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-0d280047-42fc-436b-bbb5-16b1f6d992e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:43.784938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:43.784938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:43.784938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:43.784938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:43.784938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:43.784938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:43.784938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:43.784938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:43.784938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:43.784938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:43.784938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:43.784938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:15:43.784938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:15:43.784938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:15:43.785926 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:15:43.785926 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:43.785926 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:43.785926 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:43.785926 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:43.785926 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:43.785926 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:43.785926 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:43.785926 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:43.785926 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:43.785926 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:43.785926 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:43.785926 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:43.785926 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:43.785926 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:43.785926 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:43.785926 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:43.786857 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:15:43.786857 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:15:43.786857 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:15:43.786857 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:15:43.786857 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:15:43.786857 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:15:43.786857 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:43.786857 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:43.786857 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 407/817] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:43.786857 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d280047-42fc-436b-bbb5-16b1f6d992e8 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:43.801973 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff656de-60f6-4cad-9f19-ffd3511062ab None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:43.802134 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:43.809865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:43.835920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218cf45c-58ce-4e7b-b0bf-df8fb80ea16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:43.837299 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218cf45c-58ce-4e7b-b0bf-df8fb80ea16e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:43.837580 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218cf45c-58ce-4e7b-b0bf-df8fb80ea16e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:43.837854 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218cf45c-58ce-4e7b-b0bf-df8fb80ea16e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:43.847746 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218cf45c-58ce-4e7b-b0bf-df8fb80ea16e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:43.849176 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218cf45c-58ce-4e7b-b0bf-df8fb80ea16e None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:43.866166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-218cf45c-58ce-4e7b-b0bf-df8fb80ea16e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:43.866166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:43.866166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:43.866166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:43.866166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:43.866166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:43.866166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:43.866166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:43.866166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:43.866166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:43.866166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:43.866166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:43.866166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:43.866166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:43.866166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:43.867730 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:43.867730 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:43.867730 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:43.867730 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:43.867730 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:43.867730 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:43.867730 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:43.867730 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:43.867730 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:43.867730 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:43.867730 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:43.867730 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:43.867730 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:43.867730 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:43.867730 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:43.867730 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:43.867730 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:43.868806 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:43.868806 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:43.868806 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:43.868806 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:43.871995 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 411/818] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:43.872687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218cf45c-58ce-4e7b-b0bf-df8fb80ea16e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:43.884347 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d280047-42fc-436b-bbb5-16b1f6d992e8 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:43.917597 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe3b23-9a9a-4730-b88e-2edd2c26bf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:43.918381 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe3b23-9a9a-4730-b88e-2edd2c26bf9b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:43.918662 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe3b23-9a9a-4730-b88e-2edd2c26bf9b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:43.918968 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe3b23-9a9a-4730-b88e-2edd2c26bf9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:43.925429 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe3b23-9a9a-4730-b88e-2edd2c26bf9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:43.926887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe3b23-9a9a-4730-b88e-2edd2c26bf9b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:43.940577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-f5fe3b23-9a9a-4730-b88e-2edd2c26bf9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:43.940577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:43.940577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:43.940577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:43.940577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:43.940577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:43.940577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:43.940577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:43.940577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:43.940577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:43.940577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:43.940577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:43.940577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:15:43.940577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:15:43.940577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:15:43.947664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:15:43.947664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:43.947664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:43.947664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:43.947664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:43.947664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:43.947664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:43.947664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:43.947664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:43.947664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:43.947664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:43.947664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:43.947664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:43.947664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:43.947664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:43.947664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:43.947664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:43.948658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:15:43.948658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:15:43.948658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:15:43.948658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:15:43.948658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:15:43.948658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:15:43.948658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:43.948658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:43.948658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 408/819] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:43.948658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe3b23-9a9a-4730-b88e-2edd2c26bf9b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:43.973624 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218cf45c-58ce-4e7b-b0bf-df8fb80ea16e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:44.008272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300b2b0-e6c8-4840-adb0-cfa1c331c914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:44.009056 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300b2b0-e6c8-4840-adb0-cfa1c331c914 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:44.009332 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300b2b0-e6c8-4840-adb0-cfa1c331c914 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:44.009603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300b2b0-e6c8-4840-adb0-cfa1c331c914 None admin] PATH_INFO: `/v3/services/network` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:44.010253 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2300b2b0-e6c8-4840-adb0-cfa1c331c914 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:44.011635 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2300b2b0-e6c8-4840-adb0-cfa1c331c914 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:44.017073 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:44.023119 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-2300b2b0-e6c8-4840-adb0-cfa1c331c914 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:15:44.023119 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:44.023119 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:15:44.023119 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:15:44.023119 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:15:44.023119 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:15:44.023119 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:15:44.023119 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:15:44.023119 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:15:44.023119 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:44.023119 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:15:44.023119 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:44.023119 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:44.023119 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:44.023119 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:44.023119 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:44.023119 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:44.027300 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:44.027300 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:44.027300 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:44.027300 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:44.027300 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:44.027300 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:44.027300 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:15:44.027300 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:15:44.027300 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:15:44.027300 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:15:44.027300 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:44.027300 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:44.027300 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:44.027300 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:44.027300 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:44.027300 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:44.028130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:44.028130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:44.028130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:44.028130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:44.028130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:44.028130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:44.028130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:44.028130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:44.028130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:44.028130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:44.028130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:15:44.028130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:15:44.028130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:15:44.028130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:44.028130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 412/820] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:43 2022] GET /identity/v3/services/network => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:44.028928 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300b2b0-e6c8-4840-adb0-cfa1c331c914 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:44.048682 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe3b23-9a9a-4730-b88e-2edd2c26bf9b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:44.049847 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:44.084937 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b4091-367d-435c-b5f1-70d74bc22e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:44.085603 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b4091-367d-435c-b5f1-70d74bc22e3b None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:44.085883 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b4091-367d-435c-b5f1-70d74bc22e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:44.086157 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b4091-367d-435c-b5f1-70d74bc22e3b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:44.087467 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b4091-367d-435c-b5f1-70d74bc22e3b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:44.088838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b4091-367d-435c-b5f1-70d74bc22e3b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:44.129268 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300b2b0-e6c8-4840-adb0-cfa1c331c914 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:44.162156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330fcb8-220b-44ad-b169-230f16c98b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:44.162754 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330fcb8-220b-44ad-b169-230f16c98b1a None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:44.163066 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330fcb8-220b-44ad-b169-230f16c98b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:44.163347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330fcb8-220b-44ad-b169-230f16c98b1a None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:44.164130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9330fcb8-220b-44ad-b169-230f16c98b1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:44.164508 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9330fcb8-220b-44ad-b169-230f16c98b1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:44.165939 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9330fcb8-220b-44ad-b169-230f16c98b1a None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:44.168351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 409/821] 10.222.0.24 () {60 vars in 1144 bytes} [Fri May 27 03:15:43 2022] POST /identity/v3/endpoints => generated 346 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:15:44.169252 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b4091-367d-435c-b5f1-70d74bc22e3b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:44.181739 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 413/822] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:15:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:44.182438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330fcb8-220b-44ad-b169-230f16c98b1a None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:44.285287 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:44.291109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b4091-367d-435c-b5f1-70d74bc22e3b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:44.304556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330fcb8-220b-44ad-b169-230f16c98b1a None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:44.321386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:44.330512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab561508-64b2-4729-9394-8acf768d918f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:44.331508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab561508-64b2-4729-9394-8acf768d918f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:44.331789 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab561508-64b2-4729-9394-8acf768d918f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:44.332060 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab561508-64b2-4729-9394-8acf768d918f None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:44.332796 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab561508-64b2-4729-9394-8acf768d918f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:44.333099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab561508-64b2-4729-9394-8acf768d918f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:44.334570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab561508-64b2-4729-9394-8acf768d918f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:44.344587 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf7a9d-a877-44e7-99d8-00adc34a5090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:44.345246 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf7a9d-a877-44e7-99d8-00adc34a5090 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:44.345526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf7a9d-a877-44e7-99d8-00adc34a5090 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:44.345798 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf7a9d-a877-44e7-99d8-00adc34a5090 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:44.346699 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf7a9d-a877-44e7-99d8-00adc34a5090 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:44.352687 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 410/823] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:44.352835 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf7a9d-a877-44e7-99d8-00adc34a5090 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:44.352945 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab561508-64b2-4729-9394-8acf768d918f None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:44.353018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf7a9d-a877-44e7-99d8-00adc34a5090 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:44.362338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 414/824] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:15:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:44.363303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf7a9d-a877-44e7-99d8-00adc34a5090 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:44.461008 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab561508-64b2-4729-9394-8acf768d918f None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:44.472075 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf7a9d-a877-44e7-99d8-00adc34a5090 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:44.507377 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79172a69-4535-465d-bafd-cc812f40f97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:44.508360 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79172a69-4535-465d-bafd-cc812f40f97c None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:44.508653 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79172a69-4535-465d-bafd-cc812f40f97c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:44.508926 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79172a69-4535-465d-bafd-cc812f40f97c None admin] PATH_INFO: `/v3/projects/6ceffdd955324aec84ba048d2137ae00/users/07c9688ca035484ba5341859574c2799/roles/8e315ffd561a48f0891ef53b51d548d8` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:44.513019 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225cff33-19d7-4e57-850b-34cd2a67b7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:44.513678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225cff33-19d7-4e57-850b-34cd2a67b7f0 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:44.513956 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225cff33-19d7-4e57-850b-34cd2a67b7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:44.514225 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225cff33-19d7-4e57-850b-34cd2a67b7f0 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:44.514964 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225cff33-19d7-4e57-850b-34cd2a67b7f0 None admin] RBAC: Adding query filter params (name=network) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:44.515269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225cff33-19d7-4e57-850b-34cd2a67b7f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:44.516630 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225cff33-19d7-4e57-850b-34cd2a67b7f0 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:44.525635 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 411/825] 10.222.0.24 () {56 vars in 1112 bytes} [Fri May 27 03:15:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:44.525635 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225cff33-19d7-4e57-850b-34cd2a67b7f0 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:44.526933 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:44.530050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79172a69-4535-465d-bafd-cc812f40f97c None admin] RBAC: Authorizing `identity:create_grant(project_id=6ceffdd955324aec84ba048d2137ae00, user_id=07c9688ca035484ba5341859574c2799, role_id=8e315ffd561a48f0891ef53b51d548d8)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:44.531628 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79172a69-4535-465d-bafd-cc812f40f97c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:44.574186 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:44.599334 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 415/826] 10.222.0.24 () {58 vars in 1439 bytes} [Fri May 27 03:15:44 2022] PUT /identity/v3/projects/6ceffdd955324aec84ba048d2137ae00/users/07c9688ca035484ba5341859574c2799/roles/8e315ffd561a48f0891ef53b51d548d8 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:15:44.600462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79172a69-4535-465d-bafd-cc812f40f97c None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:44.647913 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225cff33-19d7-4e57-850b-34cd2a67b7f0 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:44.701724 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b97ce5-98a2-4193-9faa-647d3fcda2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:44.702604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b97ce5-98a2-4193-9faa-647d3fcda2bb None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:44.702923 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b97ce5-98a2-4193-9faa-647d3fcda2bb None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:44.703263 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b97ce5-98a2-4193-9faa-647d3fcda2bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:44.713979 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b97ce5-98a2-4193-9faa-647d3fcda2bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:44.715710 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b97ce5-98a2-4193-9faa-647d3fcda2bb None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:44.733477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79172a69-4535-465d-bafd-cc812f40f97c None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:44.735797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-10b97ce5-98a2-4193-9faa-647d3fcda2bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:44.735797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:44.735797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:44.735797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:44.735797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:44.735797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:44.735797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:44.735797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:44.735797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:44.735797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:44.735797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:44.735797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:44.735797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:15:44.735797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:15:44.735797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:15:44.736670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:15:44.736670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:44.736670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:44.736670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:44.736670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:44.736670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:44.736670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:44.736670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:44.736670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:44.736670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:44.736670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:44.736670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:44.736670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:44.736670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:44.736670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:44.736670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:44.736670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:44.737540 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:15:44.737540 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:15:44.737540 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:15:44.737540 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:15:44.737540 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:15:44.737540 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:15:44.737540 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:44.737540 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:44.743162 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 412/827] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:44.744007 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b97ce5-98a2-4193-9faa-647d3fcda2bb None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:44.775963 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca729dc-ab69-4c25-ad2c-9f5efbbd16d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:44.776629 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca729dc-ab69-4c25-ad2c-9f5efbbd16d9 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:44.776910 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca729dc-ab69-4c25-ad2c-9f5efbbd16d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:44.777187 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca729dc-ab69-4c25-ad2c-9f5efbbd16d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:44.778029 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca729dc-ab69-4c25-ad2c-9f5efbbd16d9 None admin] RBAC: Adding query filter params (role.id=e5a908bd3ed5445b82a6053246533b88, scope.project.id=6ceffdd955324aec84ba048d2137ae00, user.id=22a9592143014f3287cd75c60ca17237) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:44.778340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca729dc-ab69-4c25-ad2c-9f5efbbd16d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:44.779845 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca729dc-ab69-4c25-ad2c-9f5efbbd16d9 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:44.791212 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 416/828] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:15:44 2022] GET /identity/v3/role_assignments?user.id=22a9592143014f3287cd75c60ca17237&scope.project.id=6ceffdd955324aec84ba048d2137ae00&role.id=e5a908bd3ed5445b82a6053246533b88 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:44.792197 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca729dc-ab69-4c25-ad2c-9f5efbbd16d9 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:44.882412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:44.908602 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b97ce5-98a2-4193-9faa-647d3fcda2bb None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:44.923293 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:44.951000 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca729dc-ab69-4c25-ad2c-9f5efbbd16d9 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:44.981455 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a154d06-4272-4463-ac71-dfdc798fbf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:44.982526 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a154d06-4272-4463-ac71-dfdc798fbf45 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:44.983032 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a154d06-4272-4463-ac71-dfdc798fbf45 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:44.983529 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a154d06-4272-4463-ac71-dfdc798fbf45 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:44.985866 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a154d06-4272-4463-ac71-dfdc798fbf45 None admin] RBAC: Adding query filter params (type=network) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:44.986392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a154d06-4272-4463-ac71-dfdc798fbf45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:44.988019 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a154d06-4272-4463-ac71-dfdc798fbf45 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:44.998877 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 413/829] 10.222.0.24 () {56 vars in 1112 bytes} [Fri May 27 03:15:44 2022] GET /identity/v3/services?type=network => generated 350 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:45.003886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a154d06-4272-4463-ac71-dfdc798fbf45 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:45.014250 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a603d46-ed59-440b-bd39-ef145050c42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:45.015737 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a603d46-ed59-440b-bd39-ef145050c42b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:45.016225 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a603d46-ed59-440b-bd39-ef145050c42b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:45.016694 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a603d46-ed59-440b-bd39-ef145050c42b None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:45.018069 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a603d46-ed59-440b-bd39-ef145050c42b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:45.018578 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a603d46-ed59-440b-bd39-ef145050c42b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:45.020892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a603d46-ed59-440b-bd39-ef145050c42b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:45.038005 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 417/830] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:15:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:45.044126 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a603d46-ed59-440b-bd39-ef145050c42b None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:45.132940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a154d06-4272-4463-ac71-dfdc798fbf45 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:45.173282 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f5204-3dcb-4e0c-9110-31fa1ca84ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:45.173952 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f5204-3dcb-4e0c-9110-31fa1ca84ab6 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:45.174237 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f5204-3dcb-4e0c-9110-31fa1ca84ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:45.174509 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f5204-3dcb-4e0c-9110-31fa1ca84ab6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:45.175820 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859f5204-3dcb-4e0c-9110-31fa1ca84ab6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:45.177425 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859f5204-3dcb-4e0c-9110-31fa1ca84ab6 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:45.182071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a603d46-ed59-440b-bd39-ef145050c42b None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:45.202463 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:45.225758 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8917e6e4-f241-4a5b-b950-8789b6ed0613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:45.226658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917e6e4-f241-4a5b-b950-8789b6ed0613 None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:45.227544 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917e6e4-f241-4a5b-b950-8789b6ed0613 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:45.227828 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8917e6e4-f241-4a5b-b950-8789b6ed0613 None admin] PATH_INFO: `/v3/projects/6ceffdd955324aec84ba048d2137ae00/users/21d559cd7127490da84185f20684929d/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:45.230035 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917e6e4-f241-4a5b-b950-8789b6ed0613 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ceffdd955324aec84ba048d2137ae00, user_id=21d559cd7127490da84185f20684929d, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:45.231713 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8917e6e4-f241-4a5b-b950-8789b6ed0613 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:45.310290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 414/831] 10.222.0.24 () {60 vars in 1144 bytes} [Fri May 27 03:15:45 2022] POST /identity/v3/endpoints => generated 327 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:15:45.352995 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 418/832] 10.222.0.24 () {58 vars in 1439 bytes} [Fri May 27 03:15:45 2022] PUT /identity/v3/projects/6ceffdd955324aec84ba048d2137ae00/users/21d559cd7127490da84185f20684929d/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:15:46.530761 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858a948-cb3c-4328-85ac-ff4c2fcce237 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:46.531282 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858a948-cb3c-4328-85ac-ff4c2fcce237 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:46.531282 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858a948-cb3c-4328-85ac-ff4c2fcce237 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:46.532882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 415/833] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:46.538140 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde827c3-2f5f-4023-aced-d7a18e1eb3c1 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:46.538523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde827c3-2f5f-4023-aced-d7a18e1eb3c1 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:46.538686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde827c3-2f5f-4023-aced-d7a18e1eb3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:46.556255 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-dde827c3-2f5f-4023-aced-d7a18e1eb3c1 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:46.557434 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-dde827c3-2f5f-4023-aced-d7a18e1eb3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:46.636293 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 419/834] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:15:46.659400 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858a948-cb3c-4328-85ac-ff4c2fcce237 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:46.694246 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:46.769516 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858a948-cb3c-4328-85ac-ff4c2fcce237 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:46.805054 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0ec08-5a8f-43cc-874d-b8b2a0c1e1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:46.805854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0ec08-5a8f-43cc-874d-b8b2a0c1e1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:46.806135 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0ec08-5a8f-43cc-874d-b8b2a0c1e1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:46.806409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0ec08-5a8f-43cc-874d-b8b2a0c1e1c4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:46.807093 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b0ec08-5a8f-43cc-874d-b8b2a0c1e1c4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:46.808502 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b0ec08-5a8f-43cc-874d-b8b2a0c1e1c4 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:46.821424 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-44b0ec08-5a8f-43cc-874d-b8b2a0c1e1c4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:15:46.821424 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:46.821424 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:15:46.821424 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:15:46.821424 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:15:46.821424 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:15:46.821424 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:15:46.821424 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:15:46.821424 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:15:46.821424 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:46.821424 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:15:46.821424 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:46.821424 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:46.821424 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:46.821424 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:46.821424 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:46.821424 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:46.822547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:46.822547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:46.822547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:46.822547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:46.822547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:46.822547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:46.822547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:15:46.822547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:15:46.822547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:15:46.822547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:15:46.822547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:46.822547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:46.822547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:46.822547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:46.822547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:46.822547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:46.823401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:46.823401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:46.823401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:46.823401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:46.823401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:46.823401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:46.823401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:46.823401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:46.823401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:46.823401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:46.823401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:15:46.823401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:15:46.823401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:15:46.823401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:46.823401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 416/835] 10.222.0.24 () {56 vars in 1114 bytes} [Fri May 27 03:15:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:46.829545 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde827c3-2f5f-4023-aced-d7a18e1eb3c1 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:46.932858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde827c3-2f5f-4023-aced-d7a18e1eb3c1 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:46.934791 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:46.968637 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef40c65-0d09-4dea-a65a-bd0c291448a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:46.969882 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef40c65-0d09-4dea-a65a-bd0c291448a5 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:46.970378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef40c65-0d09-4dea-a65a-bd0c291448a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:46.970881 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef40c65-0d09-4dea-a65a-bd0c291448a5 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:46.971798 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef40c65-0d09-4dea-a65a-bd0c291448a5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:46.972326 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef40c65-0d09-4dea-a65a-bd0c291448a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:46.973893 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef40c65-0d09-4dea-a65a-bd0c291448a5 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:46.979184 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 420/836] 10.222.0.24 () {56 vars in 1114 bytes} [Fri May 27 03:15:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:46.990327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0ec08-5a8f-43cc-874d-b8b2a0c1e1c4 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:47.018413 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca170bb-8ffc-4503-8c5d-c8468d9495c2 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:47.018705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca170bb-8ffc-4503-8c5d-c8468d9495c2 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:47.019003 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca170bb-8ffc-4503-8c5d-c8468d9495c2 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:47.022965 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 421/837] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:15:47.025721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25057af7-0b29-4bde-a714-028e8789cf92 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:47.026009 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25057af7-0b29-4bde-a714-028e8789cf92 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:47.026288 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25057af7-0b29-4bde-a714-028e8789cf92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:47.047922 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-25057af7-0b29-4bde-a714-028e8789cf92 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:47.049746 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-25057af7-0b29-4bde-a714-028e8789cf92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:47.093131 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:47.100509 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0ec08-5a8f-43cc-874d-b8b2a0c1e1c4 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:47.135352 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 422/838] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:47 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:15:47.149669 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba20013-e985-47b5-a300-c5c7cb369d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:47.150673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba20013-e985-47b5-a300-c5c7cb369d51 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:47.151169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba20013-e985-47b5-a300-c5c7cb369d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:47.151671 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba20013-e985-47b5-a300-c5c7cb369d51 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:47.152618 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba20013-e985-47b5-a300-c5c7cb369d51 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:47.153125 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba20013-e985-47b5-a300-c5c7cb369d51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:47.154916 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba20013-e985-47b5-a300-c5c7cb369d51 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:47.158435 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25057af7-0b29-4bde-a714-028e8789cf92 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:47.161258 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 417/839] 10.222.0.24 () {56 vars in 1114 bytes} [Fri May 27 03:15:46 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:47.265159 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:47.271308 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25057af7-0b29-4bde-a714-028e8789cf92 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:47.313305 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5621135-0c05-4049-9026-53964ba41699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:47.314109 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5621135-0c05-4049-9026-53964ba41699 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:47.314386 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5621135-0c05-4049-9026-53964ba41699 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:47.314658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5621135-0c05-4049-9026-53964ba41699 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:47.327196 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5621135-0c05-4049-9026-53964ba41699 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:47.331316 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6304bc-e812-4573-8870-5c02861c1489 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:47.331575 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6304bc-e812-4573-8870-5c02861c1489 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:47.331821 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6304bc-e812-4573-8870-5c02861c1489 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:47.333463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 418/840] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:15:47.335193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-f5621135-0c05-4049-9026-53964ba41699 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:47.335193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:47.335193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:47.335193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:47.335193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:47.335193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:47.335193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:47.335193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:47.335193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:47.335193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:47.335193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:47.335193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:47.335193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:47.335193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:47.335193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:47.335193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:47.336665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:47.336665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:47.336665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:47.336665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:47.336665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:47.336665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:47.336665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:47.336665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:47.336665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:47.336665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:47.336665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:47.336665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:47.336665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:47.336665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:47.336665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:47.336665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:47.337521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:47.337521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:47.337521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:47.337521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:47.337521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:47.337521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:47.337521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:47.337521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:47.337521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:47.337521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:47.337521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:15:47.337521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:47.337521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 423/841] 10.222.0.24 () {56 vars in 1102 bytes} [Fri May 27 03:15:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:15:47.342875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00227998-199e-4d1e-8738-e69c897962b4 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:47.342875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00227998-199e-4d1e-8738-e69c897962b4 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:47.342875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00227998-199e-4d1e-8738-e69c897962b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:47.348597 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5621135-0c05-4049-9026-53964ba41699 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:47.358562 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-00227998-199e-4d1e-8738-e69c897962b4 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:47.359841 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-00227998-199e-4d1e-8738-e69c897962b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:47.444931 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 419/842] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:47 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:15:47.462891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5621135-0c05-4049-9026-53964ba41699 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:47.467004 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00227998-199e-4d1e-8738-e69c897962b4 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:47.500359 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2721965-9afb-41a8-9af3-ec25cd11b2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:47.501034 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2721965-9afb-41a8-9af3-ec25cd11b2e7 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:47.501318 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2721965-9afb-41a8-9af3-ec25cd11b2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:47.501590 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2721965-9afb-41a8-9af3-ec25cd11b2e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:47.502338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2721965-9afb-41a8-9af3-ec25cd11b2e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:47.502647 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2721965-9afb-41a8-9af3-ec25cd11b2e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:47.504070 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2721965-9afb-41a8-9af3-ec25cd11b2e7 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:47.518012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 424/843] 10.222.0.24 () {56 vars in 1103 bytes} [Fri May 27 03:15:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:47.519042 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2721965-9afb-41a8-9af3-ec25cd11b2e7 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:47.539538 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:47.573146 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:47.588835 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00227998-199e-4d1e-8738-e69c897962b4 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:47.628427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2721965-9afb-41a8-9af3-ec25cd11b2e7 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:47.631152 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39740b2-36da-4306-ba73-56fdc69dc895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:47.632846 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39740b2-36da-4306-ba73-56fdc69dc895 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:47.633823 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39740b2-36da-4306-ba73-56fdc69dc895 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:47.634204 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39740b2-36da-4306-ba73-56fdc69dc895 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:47.634985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39740b2-36da-4306-ba73-56fdc69dc895 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:47.636489 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39740b2-36da-4306-ba73-56fdc69dc895 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:47.652304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-a39740b2-36da-4306-ba73-56fdc69dc895 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:15:47.652304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:47.652304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:15:47.652304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:15:47.652304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:15:47.652304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:15:47.652304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:15:47.652304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:15:47.652304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:15:47.652304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:47.652304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:15:47.652304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:47.652304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:47.652304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:47.652304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:47.652304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:47.653484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:47.653484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:47.653484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:47.653484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:47.653484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:47.653484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:47.653484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:47.653484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:15:47.653484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:15:47.653484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:15:47.653484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:15:47.653484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:47.653484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:47.653484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:47.653484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:47.653484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:47.654340 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:47.654340 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:47.654340 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:47.654340 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:47.654340 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:47.654340 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:47.654340 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:47.654340 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:47.654340 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:47.654340 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:47.654340 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:47.654340 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:15:47.654340 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:15:47.654340 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:15:47.654340 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:47.658986 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 420/844] 10.222.0.24 () {56 vars in 1132 bytes} [Fri May 27 03:15:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:47.662018 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39740b2-36da-4306-ba73-56fdc69dc895 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:47.677570 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a18416-bc19-4787-86c6-5fc0d62f0622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:47.678820 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a18416-bc19-4787-86c6-5fc0d62f0622 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:47.679330 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a18416-bc19-4787-86c6-5fc0d62f0622 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:47.679913 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a18416-bc19-4787-86c6-5fc0d62f0622 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:47.691678 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a18416-bc19-4787-86c6-5fc0d62f0622 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:47.694815 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a18416-bc19-4787-86c6-5fc0d62f0622 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:47.715160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-a2a18416-bc19-4787-86c6-5fc0d62f0622 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:47.715160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:47.715160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:47.715160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:47.715160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:47.715160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:47.715160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:47.715160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:47.715160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:47.715160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:47.715160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:47.715160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:47.715160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:47.715160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:47.715160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:47.716110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:47.716110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:47.716110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:47.716110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:47.716110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:47.716110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:47.716110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:47.716110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:47.716110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:47.716110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:47.716110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:47.716110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:47.716110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:47.716110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:47.716110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:47.716110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:47.716110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:47.716997 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:47.716997 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:47.716997 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:47.716997 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:47.723557 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 425/845] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:15:47.724725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a18416-bc19-4787-86c6-5fc0d62f0622 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:47.784366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39740b2-36da-4306-ba73-56fdc69dc895 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:47.799628 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:47.828221 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac955d08-fd40-40b1-a1e1-ec4d2810c381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:47.829043 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac955d08-fd40-40b1-a1e1-ec4d2810c381 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:47.829445 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac955d08-fd40-40b1-a1e1-ec4d2810c381 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:47.829734 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac955d08-fd40-40b1-a1e1-ec4d2810c381 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:47.830442 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac955d08-fd40-40b1-a1e1-ec4d2810c381 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:47.830745 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac955d08-fd40-40b1-a1e1-ec4d2810c381 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:47.832654 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac955d08-fd40-40b1-a1e1-ec4d2810c381 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:47.835102 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a18416-bc19-4787-86c6-5fc0d62f0622 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:47.842787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 421/846] 10.222.0.24 () {56 vars in 1126 bytes} [Fri May 27 03:15:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:47.844655 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef5a53-e93f-4c5e-9293-559fcfe68a33 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:47.844950 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef5a53-e93f-4c5e-9293-559fcfe68a33 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:47.845227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef5a53-e93f-4c5e-9293-559fcfe68a33 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:47.847025 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 422/847] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:47.849163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef5a53-e93f-4c5e-9293-559fcfe68a33 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:47.870026 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:47.873322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85f31e-ed26-4a3f-8edd-d944ed44440d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:47.873997 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85f31e-ed26-4a3f-8edd-d944ed44440d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:47.874280 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85f31e-ed26-4a3f-8edd-d944ed44440d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:47.874557 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85f31e-ed26-4a3f-8edd-d944ed44440d None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:47.875322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85f31e-ed26-4a3f-8edd-d944ed44440d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:47.875630 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85f31e-ed26-4a3f-8edd-d944ed44440d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:47.876999 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85f31e-ed26-4a3f-8edd-d944ed44440d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:47.889087 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 426/848] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:47.890595 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82874332-a673-4014-a5f0-cd066d5b5111 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:47.890935 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82874332-a673-4014-a5f0-cd066d5b5111 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:47.891188 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82874332-a673-4014-a5f0-cd066d5b5111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:47.908179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-82874332-a673-4014-a5f0-cd066d5b5111 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:47.909353 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-82874332-a673-4014-a5f0-cd066d5b5111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:47.965043 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef5a53-e93f-4c5e-9293-559fcfe68a33 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:47.979883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 427/849] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:47 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:15:47.980630 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82874332-a673-4014-a5f0-cd066d5b5111 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:48.007217 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891bf62-5fcf-4bed-adf3-4da3bc73cf57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:48.008248 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891bf62-5fcf-4bed-adf3-4da3bc73cf57 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:48.008535 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891bf62-5fcf-4bed-adf3-4da3bc73cf57 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:48.008816 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0891bf62-5fcf-4bed-adf3-4da3bc73cf57 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:48.009531 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891bf62-5fcf-4bed-adf3-4da3bc73cf57 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:48.009842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891bf62-5fcf-4bed-adf3-4da3bc73cf57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:48.011391 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0891bf62-5fcf-4bed-adf3-4da3bc73cf57 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:48.016453 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 423/850] 10.222.0.24 () {56 vars in 1126 bytes} [Fri May 27 03:15:47 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:48.017129 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891bf62-5fcf-4bed-adf3-4da3bc73cf57 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:48.057332 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:48.114379 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82874332-a673-4014-a5f0-cd066d5b5111 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:48.131013 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:48.156195 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b423ef7-e249-4712-9832-43a0980edfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:48.157354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b423ef7-e249-4712-9832-43a0980edfc9 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:48.157464 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0891bf62-5fcf-4bed-adf3-4da3bc73cf57 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:48.157910 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b423ef7-e249-4712-9832-43a0980edfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:48.158354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b423ef7-e249-4712-9832-43a0980edfc9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:48.168830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b423ef7-e249-4712-9832-43a0980edfc9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:48.171026 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b423ef7-e249-4712-9832-43a0980edfc9 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:48.191891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-1b423ef7-e249-4712-9832-43a0980edfc9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:15:48.191891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:48.191891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:48.191891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:48.191891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:48.191891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:48.191891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:48.191891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:48.191891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:48.191891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:48.191891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:48.191891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:48.191891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:15:48.191891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:15:48.191891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:15:48.191891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:15:48.192918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:48.192918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:48.192918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:15:48.192918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:48.192918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:15:48.192918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:48.192918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:48.192918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:48.192918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:48.192918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:48.192918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:48.192918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:48.192918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:48.192918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:48.192918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:48.192918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:48.192918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:48.192918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:48.193858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:48.193858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:48.193858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:15:48.193858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:15:48.193858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:15:48.193858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:15:48.193858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:15:48.193858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:15:48.193858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:15:48.193858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:48.198643 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 428/851] 10.222.0.24 () {56 vars in 1117 bytes} [Fri May 27 03:15:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:48.199660 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b423ef7-e249-4712-9832-43a0980edfc9 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:48.206333 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9bea2f-8e1b-4de2-9fe5-edf95d208539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:48.207464 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9bea2f-8e1b-4de2-9fe5-edf95d208539 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:48.207923 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9bea2f-8e1b-4de2-9fe5-edf95d208539 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:48.208367 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9bea2f-8e1b-4de2-9fe5-edf95d208539 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:48.220208 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9bea2f-8e1b-4de2-9fe5-edf95d208539 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:48.228769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-ee9bea2f-8e1b-4de2-9fe5-edf95d208539 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:15:48.228769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:48.228769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:48.228769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:48.228769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:48.228769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:48.228769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:48.228769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:48.228769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:48.228769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:48.228769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:48.228769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:48.228769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:48.228769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:48.228769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:48.228769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:48.229680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:48.229680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:48.229680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:48.229680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:48.229680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:48.229680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:48.229680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:48.229680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:48.229680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:48.229680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:48.229680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:48.229680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:48.229680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:48.229680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:48.229680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:48.229680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:48.230623 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:48.230623 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:48.230623 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:48.230623 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:48.230623 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:48.230623 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:48.230623 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:48.230623 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:48.230623 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:48.230623 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:48.230623 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:15:48.230623 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:48.233575 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 424/852] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:15:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:48.238884 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9bea2f-8e1b-4de2-9fe5-edf95d208539 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:48.306113 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b423ef7-e249-4712-9832-43a0980edfc9 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:48.329812 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:48.346675 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a311dd-92fa-45db-a7df-8388b9507b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:48.347376 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a311dd-92fa-45db-a7df-8388b9507b30 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:48.347661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a311dd-92fa-45db-a7df-8388b9507b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:48.348629 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a311dd-92fa-45db-a7df-8388b9507b30 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:48.349518 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a311dd-92fa-45db-a7df-8388b9507b30 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:48.349831 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a311dd-92fa-45db-a7df-8388b9507b30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:48.351308 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a311dd-92fa-45db-a7df-8388b9507b30 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:48.353688 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9bea2f-8e1b-4de2-9fe5-edf95d208539 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:48.375452 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 429/853] 10.222.0.24 () {56 vars in 1149 bytes} [Fri May 27 03:15:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:48.376381 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a311dd-92fa-45db-a7df-8388b9507b30 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:48.392609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8393ee0-84c8-48f2-8597-202f2492369b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:48.393603 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8393ee0-84c8-48f2-8597-202f2492369b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:48.394131 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8393ee0-84c8-48f2-8597-202f2492369b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:48.394663 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8393ee0-84c8-48f2-8597-202f2492369b None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:48.395866 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8393ee0-84c8-48f2-8597-202f2492369b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:48.396393 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8393ee0-84c8-48f2-8597-202f2492369b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:48.397952 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8393ee0-84c8-48f2-8597-202f2492369b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:48.408025 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 425/854] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:15:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:48.414977 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8393ee0-84c8-48f2-8597-202f2492369b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:48.419135 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:48.481530 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a311dd-92fa-45db-a7df-8388b9507b30 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:48.517660 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e8eec-ed47-4957-890f-faa72ea99433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:48.518312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e8eec-ed47-4957-890f-faa72ea99433 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:48.518618 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e8eec-ed47-4957-890f-faa72ea99433 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:48.519007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e8eec-ed47-4957-890f-faa72ea99433 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:48.521015 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8393ee0-84c8-48f2-8597-202f2492369b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:48.571780 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699936f8-8abe-496d-b2e7-76b8c451895f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:48.572326 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:48.573716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699936f8-8abe-496d-b2e7-76b8c451895f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:48.574099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699936f8-8abe-496d-b2e7-76b8c451895f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:48.574306 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699936f8-8abe-496d-b2e7-76b8c451895f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:48.585937 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699936f8-8abe-496d-b2e7-76b8c451895f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:48.587941 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699936f8-8abe-496d-b2e7-76b8c451895f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:48.595352 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2e8eec-ed47-4957-890f-faa72ea99433 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:48.596811 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2e8eec-ed47-4957-890f-faa72ea99433 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:48.607232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-699936f8-8abe-496d-b2e7-76b8c451895f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:48.607232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:48.607232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:48.607232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:48.607232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:48.607232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:48.607232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:48.607232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:48.607232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:48.607232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:48.607232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:48.607232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:48.607232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:48.607232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:48.607232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:48.608116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:48.608116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:48.608116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:48.608116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:48.608116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:48.608116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:48.608116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:48.608116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:48.608116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:48.608116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:48.608116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:48.608116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:48.608116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:48.608116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:48.608116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:48.608116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:48.608116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:48.609018 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:48.609018 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:48.609018 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:48.609018 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:48.609018 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 426/855] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:48.613940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699936f8-8abe-496d-b2e7-76b8c451895f None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:48.673171 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:48.711734 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 430/856] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:15:48 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 03:15:48.718120 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e8eec-ed47-4957-890f-faa72ea99433 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:48.719587 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699936f8-8abe-496d-b2e7-76b8c451895f None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:48.757370 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df931d-b3e4-4eba-ac44-39c031d19b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:48.758251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df931d-b3e4-4eba-ac44-39c031d19b95 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:48.758602 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df931d-b3e4-4eba-ac44-39c031d19b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:48.758990 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df931d-b3e4-4eba-ac44-39c031d19b95 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:48.759809 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df931d-b3e4-4eba-ac44-39c031d19b95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:48.760188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df931d-b3e4-4eba-ac44-39c031d19b95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:48.761649 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df931d-b3e4-4eba-ac44-39c031d19b95 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:48.774744 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 427/857] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:15:48.780943 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df931d-b3e4-4eba-ac44-39c031d19b95 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:48.812515 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:48.835238 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e8eec-ed47-4957-890f-faa72ea99433 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:48.872062 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70490924-639b-45a9-95a0-0e0d3a225143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:48.872913 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70490924-639b-45a9-95a0-0e0d3a225143 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:48.873191 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70490924-639b-45a9-95a0-0e0d3a225143 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:48.873513 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70490924-639b-45a9-95a0-0e0d3a225143 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:48.884904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70490924-639b-45a9-95a0-0e0d3a225143 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:48.886343 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70490924-639b-45a9-95a0-0e0d3a225143 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:48.905220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-70490924-639b-45a9-95a0-0e0d3a225143 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:48.905220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:48.905220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:48.905220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:48.905220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:48.905220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:48.905220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:48.905220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:48.905220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:48.905220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:48.905220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:48.905220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:48.905220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:48.905220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:48.905220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:48.906149 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:48.906149 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:48.906149 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:48.906149 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:48.906149 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:48.906149 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:48.906149 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:48.906149 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:48.906149 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:48.906149 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:48.906149 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:48.906149 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:48.906149 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:48.906149 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:48.906149 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:48.906149 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:48.906149 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:48.907078 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:48.907078 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:48.907078 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:48.907078 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:48.907078 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 431/858] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:48.911837 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70490924-639b-45a9-95a0-0e0d3a225143 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:48.922228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df931d-b3e4-4eba-ac44-39c031d19b95 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:48.938612 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:48.960906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9cff70-601d-4db2-bb6c-cbed4fcdf1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:48.962163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9cff70-601d-4db2-bb6c-cbed4fcdf1cc None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:48.962706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9cff70-601d-4db2-bb6c-cbed4fcdf1cc None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:48.963241 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9cff70-601d-4db2-bb6c-cbed4fcdf1cc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:48.972676 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9cff70-601d-4db2-bb6c-cbed4fcdf1cc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:48.974492 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9cff70-601d-4db2-bb6c-cbed4fcdf1cc None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:48.991514 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-5c9cff70-601d-4db2-bb6c-cbed4fcdf1cc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 03:15:48.991514 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:48.991514 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:48.991514 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:48.991514 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:48.991514 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:48.991514 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:48.991514 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:48.991514 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:48.991514 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:48.991514 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:48.991514 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:48.991514 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:15:48.991514 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:15:48.991514 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:15:48.991514 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:15:48.992584 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:48.992584 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:48.992584 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:15:48.992584 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:48.992584 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:15:48.992584 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:48.992584 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:48.992584 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:48.992584 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:48.992584 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:48.992584 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:48.992584 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:48.992584 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:48.992584 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:48.992584 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:48.992584 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:48.992584 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:48.992584 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:48.993558 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:48.993558 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:48.993558 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:15:48.993558 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:15:48.993558 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:15:48.993558 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:15:48.993558 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:15:48.993558 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:15:48.993558 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 03:15:48.993558 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:48.996335 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 428/859] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:15:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:49.006816 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9cff70-601d-4db2-bb6c-cbed4fcdf1cc None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:49.035336 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70490924-639b-45a9-95a0-0e0d3a225143 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:49.083173 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:49.089368 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73db07-d66a-4d0a-bd86-d78ca4de7b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:49.090221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73db07-d66a-4d0a-bd86-d78ca4de7b51 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:49.090648 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73db07-d66a-4d0a-bd86-d78ca4de7b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:49.091543 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73db07-d66a-4d0a-bd86-d78ca4de7b51 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:49.096944 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c73db07-d66a-4d0a-bd86-d78ca4de7b51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:49.096944 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c73db07-d66a-4d0a-bd86-d78ca4de7b51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:49.096944 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c73db07-d66a-4d0a-bd86-d78ca4de7b51 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:49.110912 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 432/860] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:15:49.115782 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73db07-d66a-4d0a-bd86-d78ca4de7b51 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:49.125623 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9cff70-601d-4db2-bb6c-cbed4fcdf1cc None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:49.163625 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e4f7c4-9670-411a-b7e7-27eb2ee8e208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:49.163857 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4f7c4-9670-411a-b7e7-27eb2ee8e208 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:49.164680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4f7c4-9670-411a-b7e7-27eb2ee8e208 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:49.164958 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4f7c4-9670-411a-b7e7-27eb2ee8e208 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:49.165798 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e4f7c4-9670-411a-b7e7-27eb2ee8e208 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:49.166163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e4f7c4-9670-411a-b7e7-27eb2ee8e208 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:49.167631 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e4f7c4-9670-411a-b7e7-27eb2ee8e208 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:49.189817 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 429/861] 10.222.0.24 () {56 vars in 1135 bytes} [Fri May 27 03:15:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:49.196147 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e4f7c4-9670-411a-b7e7-27eb2ee8e208 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:49.244564 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:49.256567 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73db07-d66a-4d0a-bd86-d78ca4de7b51 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:49.306757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07566d9-5f96-4732-a803-da36ee5a1c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:49.307832 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07566d9-5f96-4732-a803-da36ee5a1c83 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:49.308806 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07566d9-5f96-4732-a803-da36ee5a1c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:49.309238 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07566d9-5f96-4732-a803-da36ee5a1c83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:49.317559 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07566d9-5f96-4732-a803-da36ee5a1c83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:49.319227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07566d9-5f96-4732-a803-da36ee5a1c83 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:49.322041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e4f7c4-9670-411a-b7e7-27eb2ee8e208 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:49.334610 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-c07566d9-5f96-4732-a803-da36ee5a1c83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:49.334610 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:49.334610 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:49.334610 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:49.334610 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:49.334610 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:49.334610 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:49.334610 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:49.334610 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:49.334610 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:49.334610 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:49.334610 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:49.334610 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:15:49.334610 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:15:49.334610 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:15:49.335764 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:15:49.335764 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:49.335764 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:49.335764 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:49.335764 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:49.335764 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:49.335764 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:49.335764 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:49.335764 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:49.335764 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:49.335764 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:49.335764 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:49.335764 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:49.335764 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:49.335764 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:49.335764 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:49.335764 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:49.337415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:15:49.337415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:15:49.337415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:15:49.337415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:15:49.337415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:15:49.337415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:15:49.337415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:49.337415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:49.339569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 433/862] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:49.347764 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07566d9-5f96-4732-a803-da36ee5a1c83 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:49.362337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8473cb-9322-4eff-a5b1-6273a9d5a9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:49.363177 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8473cb-9322-4eff-a5b1-6273a9d5a9f7 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:49.363553 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8473cb-9322-4eff-a5b1-6273a9d5a9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:49.363923 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8473cb-9322-4eff-a5b1-6273a9d5a9f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:49.416931 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:49.434361 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8473cb-9322-4eff-a5b1-6273a9d5a9f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:49.435895 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8473cb-9322-4eff-a5b1-6273a9d5a9f7 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:49.465998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:49.477309 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07566d9-5f96-4732-a803-da36ee5a1c83 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:49.512961 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740b344-bcb8-40ed-8ee9-2f676bcee04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:49.513810 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740b344-bcb8-40ed-8ee9-2f676bcee04d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:49.514156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740b344-bcb8-40ed-8ee9-2f676bcee04d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:49.514499 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740b344-bcb8-40ed-8ee9-2f676bcee04d None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:49.515504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e740b344-bcb8-40ed-8ee9-2f676bcee04d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:49.515880 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e740b344-bcb8-40ed-8ee9-2f676bcee04d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:49.517489 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e740b344-bcb8-40ed-8ee9-2f676bcee04d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:49.535521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 434/863] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:15:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:49.535521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372be62a-fed6-4711-b790-a2b24bbabdb1 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:49.535521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372be62a-fed6-4711-b790-a2b24bbabdb1 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:49.535521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372be62a-fed6-4711-b790-a2b24bbabdb1 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:49.535521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 435/864] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:49.537584 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372be62a-fed6-4711-b790-a2b24bbabdb1 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:49.551175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 430/865] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:15:49 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:15:49.553528 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b592a6-8462-4596-b962-cee321e18708 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:49.553919 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b592a6-8462-4596-b962-cee321e18708 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:49.554431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b592a6-8462-4596-b962-cee321e18708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:49.573361 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-a5b592a6-8462-4596-b962-cee321e18708 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:49.574625 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a5b592a6-8462-4596-b962-cee321e18708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:49.643407 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 431/866] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:49 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:15:49.644236 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b592a6-8462-4596-b962-cee321e18708 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:49.655962 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372be62a-fed6-4711-b790-a2b24bbabdb1 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:49.670943 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:49.704797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:49.707035 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281ef6d-cfcf-4abb-a3cb-f4196b4a6814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:49.707693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281ef6d-cfcf-4abb-a3cb-f4196b4a6814 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:49.707979 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281ef6d-cfcf-4abb-a3cb-f4196b4a6814 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:49.708300 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281ef6d-cfcf-4abb-a3cb-f4196b4a6814 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:49.709245 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281ef6d-cfcf-4abb-a3cb-f4196b4a6814 None admin] RBAC: Adding query filter params (role.id=8e315ffd561a48f0891ef53b51d548d8, scope.project.id=6ceffdd955324aec84ba048d2137ae00, user.id=07c9688ca035484ba5341859574c2799) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:49.710014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281ef6d-cfcf-4abb-a3cb-f4196b4a6814 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:49.711846 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281ef6d-cfcf-4abb-a3cb-f4196b4a6814 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:49.719106 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 436/867] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:15:49 2022] GET /identity/v3/role_assignments?user.id=07c9688ca035484ba5341859574c2799&scope.project.id=6ceffdd955324aec84ba048d2137ae00&role.id=8e315ffd561a48f0891ef53b51d548d8 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:49.720144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281ef6d-cfcf-4abb-a3cb-f4196b4a6814 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:49.757388 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b592a6-8462-4596-b962-cee321e18708 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:49.811201 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e2b51-364b-4c60-b1ad-6a2f8fc3dde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:49.812328 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e2b51-364b-4c60-b1ad-6a2f8fc3dde5 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:49.812785 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e2b51-364b-4c60-b1ad-6a2f8fc3dde5 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:49.813234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e2b51-364b-4c60-b1ad-6a2f8fc3dde5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:49.829210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e2b51-364b-4c60-b1ad-6a2f8fc3dde5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:49.831838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e2b51-364b-4c60-b1ad-6a2f8fc3dde5 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:49.845269 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281ef6d-cfcf-4abb-a3cb-f4196b4a6814 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:49.855337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-b27e2b51-364b-4c60-b1ad-6a2f8fc3dde5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:49.855337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:49.855337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:49.855337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:49.855337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:49.855337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:49.855337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:49.855337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:49.855337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:49.855337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:49.855337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:49.855337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:49.855337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:49.855337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:49.855337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:49.856286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:49.856286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:49.856286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:49.856286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:49.856286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:49.856286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:49.856286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:49.856286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:49.856286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:49.856286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:49.856286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:49.856286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:49.856286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:49.856286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:49.856286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:49.856286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:49.856286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:49.857149 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:49.857149 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:49.857149 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:49.857149 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:49.859699 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 432/868] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:49.865013 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e2b51-364b-4c60-b1ad-6a2f8fc3dde5 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:49.903450 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82581cb2-6a7f-4215-9a5c-ea2849e8c6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:49.904527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82581cb2-6a7f-4215-9a5c-ea2849e8c6ed None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:49.905050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82581cb2-6a7f-4215-9a5c-ea2849e8c6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:49.905582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82581cb2-6a7f-4215-9a5c-ea2849e8c6ed None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:49.906716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82581cb2-6a7f-4215-9a5c-ea2849e8c6ed None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:49.908508 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82581cb2-6a7f-4215-9a5c-ea2849e8c6ed None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:49.993851 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 437/869] 10.222.0.24 () {60 vars in 1141 bytes} [Fri May 27 03:15:49 2022] POST /identity/v3/services => generated 251 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:15:49.998676 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e2b51-364b-4c60-b1ad-6a2f8fc3dde5 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:50.016668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:50.046754 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b615d5f-3948-406d-91e7-cb3b894f94c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:50.047529 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b615d5f-3948-406d-91e7-cb3b894f94c4 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:50.047805 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b615d5f-3948-406d-91e7-cb3b894f94c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:50.048073 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b615d5f-3948-406d-91e7-cb3b894f94c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:50.048962 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b615d5f-3948-406d-91e7-cb3b894f94c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:50.049330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b615d5f-3948-406d-91e7-cb3b894f94c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:50.050878 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b615d5f-3948-406d-91e7-cb3b894f94c4 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:50.065294 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 433/870] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:50.071301 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82581cb2-6a7f-4215-9a5c-ea2849e8c6ed None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:50.113791 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:50.194932 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82581cb2-6a7f-4215-9a5c-ea2849e8c6ed None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:50.231562 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e3eb5c-524e-4e42-8b0d-6792578e0f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:50.232373 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3eb5c-524e-4e42-8b0d-6792578e0f87 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:50.232656 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3eb5c-524e-4e42-8b0d-6792578e0f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:50.232933 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3eb5c-524e-4e42-8b0d-6792578e0f87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:50.240470 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e3eb5c-524e-4e42-8b0d-6792578e0f87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:50.241960 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e3eb5c-524e-4e42-8b0d-6792578e0f87 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:50.259181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-81e3eb5c-524e-4e42-8b0d-6792578e0f87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:50.259181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:50.259181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:50.259181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:50.259181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:50.259181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:50.259181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:50.259181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:50.259181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:50.259181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:50.259181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:50.259181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:50.259181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:15:50.259181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:15:50.259181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:15:50.260194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:15:50.260194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:50.260194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:50.260194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:50.260194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:50.260194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:50.260194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:50.260194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:50.260194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:50.260194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:50.260194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:50.260194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:50.260194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:50.260194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:50.260194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:50.260194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:50.260194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:50.261160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:15:50.261160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:15:50.261160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:15:50.261160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:15:50.261160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:15:50.261160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:15:50.261160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:50.261160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:50.261160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 438/871] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:50.267637 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b615d5f-3948-406d-91e7-cb3b894f94c4 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:50.388530 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b615d5f-3948-406d-91e7-cb3b894f94c4 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:50.423269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc72622-d993-4476-ba28-8262a58e9852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:50.423929 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc72622-d993-4476-ba28-8262a58e9852 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:50.424212 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc72622-d993-4476-ba28-8262a58e9852 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:50.424484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc72622-d993-4476-ba28-8262a58e9852 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:50.425283 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc72622-d993-4476-ba28-8262a58e9852 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:50.425594 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc72622-d993-4476-ba28-8262a58e9852 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:50.427049 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc72622-d993-4476-ba28-8262a58e9852 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:50.439877 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 434/872] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:15:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:15:50.446267 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e3eb5c-524e-4e42-8b0d-6792578e0f87 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:50.454182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74512467-cd57-4ff0-8d1a-6323fa54fd56 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:50.454461 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74512467-cd57-4ff0-8d1a-6323fa54fd56 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:50.454740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74512467-cd57-4ff0-8d1a-6323fa54fd56 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:50.456369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 435/873] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:15:50.461467 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d4191-f3d7-47be-9860-f599442d0f39 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:50.461694 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d4191-f3d7-47be-9860-f599442d0f39 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:50.461954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d4191-f3d7-47be-9860-f599442d0f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:50.478627 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-eb2d4191-f3d7-47be-9860-f599442d0f39 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:50.479839 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-eb2d4191-f3d7-47be-9860-f599442d0f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:50.503983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:50.551986 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:50.557785 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 436/874] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:50 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:15:50.559852 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e3eb5c-524e-4e42-8b0d-6792578e0f87 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:50.580095 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d4191-f3d7-47be-9860-f599442d0f39 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:50.597181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa68176-6291-43d7-b5d4-2e9c0f407c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:50.597951 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa68176-6291-43d7-b5d4-2e9c0f407c33 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:50.598234 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa68176-6291-43d7-b5d4-2e9c0f407c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:50.598619 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa68176-6291-43d7-b5d4-2e9c0f407c33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:50.599495 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa68176-6291-43d7-b5d4-2e9c0f407c33 None admin] RBAC: Adding query filter params (role.id=e5a908bd3ed5445b82a6053246533b88, scope.project.id=6ceffdd955324aec84ba048d2137ae00, user.id=21d559cd7127490da84185f20684929d) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:50.599807 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa68176-6291-43d7-b5d4-2e9c0f407c33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:50.601238 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa68176-6291-43d7-b5d4-2e9c0f407c33 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:50.608264 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 439/875] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:15:50 2022] GET /identity/v3/role_assignments?user.id=21d559cd7127490da84185f20684929d&scope.project.id=6ceffdd955324aec84ba048d2137ae00&role.id=e5a908bd3ed5445b82a6053246533b88 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:50.689151 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d4191-f3d7-47be-9860-f599442d0f39 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:50.727238 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecdea47-e924-4eb3-9df9-02cc5a05ce6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:50.728250 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdea47-e924-4eb3-9df9-02cc5a05ce6d None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:50.728725 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdea47-e924-4eb3-9df9-02cc5a05ce6d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:50.729184 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecdea47-e924-4eb3-9df9-02cc5a05ce6d None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:50.730108 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecdea47-e924-4eb3-9df9-02cc5a05ce6d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:50.731709 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecdea47-e924-4eb3-9df9-02cc5a05ce6d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:50.863163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 437/876] 10.222.0.24 () {60 vars in 1141 bytes} [Fri May 27 03:15:50 2022] POST /identity/v3/services => generated 269 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:15:52.150161 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1e595-9a33-40ed-bc0c-19eab1f0db76 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:52.150707 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1e595-9a33-40ed-bc0c-19eab1f0db76 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:52.150707 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1e595-9a33-40ed-bc0c-19eab1f0db76 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:52.152708 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 440/877] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:52.158605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719b5b3-8090-4675-ab89-4c94b5e19a65 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:52.159180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719b5b3-8090-4675-ab89-4c94b5e19a65 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:52.159180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719b5b3-8090-4675-ab89-4c94b5e19a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:52.177179 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-9719b5b3-8090-4675-ab89-4c94b5e19a65 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:52.178454 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-9719b5b3-8090-4675-ab89-4c94b5e19a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:52.193470 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:52.252967 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 438/878] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:52 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:15:52.276737 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1e595-9a33-40ed-bc0c-19eab1f0db76 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:52.338156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090cf94-55a8-4b39-9fef-4d59649cf5a3 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:52.338339 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090cf94-55a8-4b39-9fef-4d59649cf5a3 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:52.338620 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090cf94-55a8-4b39-9fef-4d59649cf5a3 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:52.340238 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 439/879] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:15:52.345482 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590ab29-4637-4b4d-bc9c-9efe9f40807f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:52.345992 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590ab29-4637-4b4d-bc9c-9efe9f40807f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:52.346475 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590ab29-4637-4b4d-bc9c-9efe9f40807f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:52.363854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-b590ab29-4637-4b4d-bc9c-9efe9f40807f None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:52.365115 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b590ab29-4637-4b4d-bc9c-9efe9f40807f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:52.385934 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1e595-9a33-40ed-bc0c-19eab1f0db76 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:52.422200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd57082-1fec-44f2-bd1b-ffd83b435abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:52.423362 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd57082-1fec-44f2-bd1b-ffd83b435abb None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:52.423850 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd57082-1fec-44f2-bd1b-ffd83b435abb None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:52.424306 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd57082-1fec-44f2-bd1b-ffd83b435abb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:52.436061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd57082-1fec-44f2-bd1b-ffd83b435abb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:52.444571 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-cbd57082-1fec-44f2-bd1b-ffd83b435abb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:15:52.444571 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:52.444571 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:52.444571 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:52.444571 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:52.444571 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:52.444571 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:52.444571 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:52.444571 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:52.444571 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:52.444571 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:52.444571 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:52.444571 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:52.444571 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:52.444571 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:52.444571 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:52.445564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:52.445564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:52.445564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:52.445564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:52.445564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:52.445564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:52.445564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:52.445564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:52.445564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:52.445564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:52.445564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:52.445564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:52.445564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:52.445564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:52.445564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:52.445564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:52.446426 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:52.446426 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:52.446426 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:52.446426 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:52.446426 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:52.446426 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:52.446426 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:52.446426 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:52.446426 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:52.446426 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:52.446426 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:15:52.446426 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:52.451978 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 440/880] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:52 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:15:52.452104 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 441/881] 10.222.0.24 () {56 vars in 1120 bytes} [Fri May 27 03:15:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:52.457613 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590ab29-4637-4b4d-bc9c-9efe9f40807f None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:52.472171 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd57082-1fec-44f2-bd1b-ffd83b435abb None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:52.493363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:52.569195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:52.591394 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd57082-1fec-44f2-bd1b-ffd83b435abb None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:52.591998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590ab29-4637-4b4d-bc9c-9efe9f40807f None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:52.627503 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be12e5-d40b-4d3a-afee-b61106ad004a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:52.628540 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be12e5-d40b-4d3a-afee-b61106ad004a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:52.629023 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be12e5-d40b-4d3a-afee-b61106ad004a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:52.629563 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be12e5-d40b-4d3a-afee-b61106ad004a None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:52.630572 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be12e5-d40b-4d3a-afee-b61106ad004a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:52.631125 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be12e5-d40b-4d3a-afee-b61106ad004a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:52.633285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be12e5-d40b-4d3a-afee-b61106ad004a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:52.638797 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a260ec4-3fba-4b3d-afb1-e7df11f53bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:52.640070 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a260ec4-3fba-4b3d-afb1-e7df11f53bca None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:52.640361 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a260ec4-3fba-4b3d-afb1-e7df11f53bca None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:52.640637 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a260ec4-3fba-4b3d-afb1-e7df11f53bca None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:52.641447 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a260ec4-3fba-4b3d-afb1-e7df11f53bca None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:52.643043 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a260ec4-3fba-4b3d-afb1-e7df11f53bca None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:52.644297 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 441/882] 10.222.0.24 () {56 vars in 1115 bytes} [Fri May 27 03:15:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:52.651428 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be12e5-d40b-4d3a-afee-b61106ad004a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:52.663556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-9a260ec4-3fba-4b3d-afb1-e7df11f53bca None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:15:52.663556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:52.663556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:15:52.663556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:15:52.663556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:15:52.663556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:15:52.663556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:15:52.663556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:15:52.663556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:15:52.663556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:52.663556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:15:52.663556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:52.663556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:52.663556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:52.663556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:52.663556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:52.663556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:52.665272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:52.665272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:52.665272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:52.665272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:52.665272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:52.665272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:52.665272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:15:52.665272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:15:52.665272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:15:52.665272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:15:52.665272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:52.665272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:52.665272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:52.665272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:52.665272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:52.665272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:52.666281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:52.666281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:52.666281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:52.666281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:52.666281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:52.666281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:52.666281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:52.666281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:52.666281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:52.666281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:52.666281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:15:52.666281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:15:52.666281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:15:52.666281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:52.674503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 442/883] 10.222.0.24 () {56 vars in 1114 bytes} [Fri May 27 03:15:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:15:52.676348 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a260ec4-3fba-4b3d-afb1-e7df11f53bca None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:52.790949 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:52.791511 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be12e5-d40b-4d3a-afee-b61106ad004a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:52.814893 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a260ec4-3fba-4b3d-afb1-e7df11f53bca None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:52.837297 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f73e18-148d-441e-87ca-d620d8bdb2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:52.838148 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f73e18-148d-441e-87ca-d620d8bdb2a6 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:52.838430 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f73e18-148d-441e-87ca-d620d8bdb2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:52.838704 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f73e18-148d-441e-87ca-d620d8bdb2a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:52.847879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:52.853345 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f73e18-148d-441e-87ca-d620d8bdb2a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:52.855462 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f73e18-148d-441e-87ca-d620d8bdb2a6 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:52.860366 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff97a0f-e3d0-4c13-9314-e7f2d5ce9215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:52.863773 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff97a0f-e3d0-4c13-9314-e7f2d5ce9215 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:52.863773 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff97a0f-e3d0-4c13-9314-e7f2d5ce9215 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:52.863773 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff97a0f-e3d0-4c13-9314-e7f2d5ce9215 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:52.863773 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff97a0f-e3d0-4c13-9314-e7f2d5ce9215 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:52.863773 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff97a0f-e3d0-4c13-9314-e7f2d5ce9215 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:52.865776 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff97a0f-e3d0-4c13-9314-e7f2d5ce9215 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:52.875459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-b4f73e18-148d-441e-87ca-d620d8bdb2a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:52.875459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:52.875459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:52.875459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:52.875459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:52.875459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:52.875459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:52.875459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:52.875459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:52.875459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:52.875459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:52.875459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:52.875459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:52.875459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:52.875459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:52.876604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:52.876604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:52.876604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:52.876604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:52.876604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:52.876604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:52.876604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:52.876604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:52.876604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:52.876604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:52.876604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:52.876604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:52.876604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:52.876604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:52.876604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:52.876604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:52.876604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:52.877654 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:52.877654 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:52.877654 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:52.877654 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:52.878048 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 443/885] 10.222.0.24 () {56 vars in 1114 bytes} [Fri May 27 03:15:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:52.879506 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff97a0f-e3d0-4c13-9314-e7f2d5ce9215 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:52.880611 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 442/885] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:15:52.881566 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f73e18-148d-441e-87ca-d620d8bdb2a6 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:53.002787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f73e18-148d-441e-87ca-d620d8bdb2a6 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:53.008784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff97a0f-e3d0-4c13-9314-e7f2d5ce9215 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:53.041335 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf3e3a8-9cf9-466e-948a-7a9badba6a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:53.042975 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3e3a8-9cf9-466e-948a-7a9badba6a6e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:53.043566 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3e3a8-9cf9-466e-948a-7a9badba6a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:53.044001 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf3e3a8-9cf9-466e-948a-7a9badba6a6e None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:53.044896 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf3e3a8-9cf9-466e-948a-7a9badba6a6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:53.045532 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf3e3a8-9cf9-466e-948a-7a9badba6a6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:53.046561 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4895000d-7c79-4daa-a6a5-742ab6105fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:53.047244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895000d-7c79-4daa-a6a5-742ab6105fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:53.050289 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf3e3a8-9cf9-466e-948a-7a9badba6a6e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:53.050407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895000d-7c79-4daa-a6a5-742ab6105fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:53.050407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895000d-7c79-4daa-a6a5-742ab6105fc6 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:53.050407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4895000d-7c79-4daa-a6a5-742ab6105fc6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:53.050407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4895000d-7c79-4daa-a6a5-742ab6105fc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:53.050407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4895000d-7c79-4daa-a6a5-742ab6105fc6 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:53.055712 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 444/886] 10.222.0.24 () {56 vars in 1114 bytes} [Fri May 27 03:15:52 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:15:53.056952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda89f6-3839-452c-a01e-163ff7392524 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:53.057230 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda89f6-3839-452c-a01e-163ff7392524 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:53.057499 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda89f6-3839-452c-a01e-163ff7392524 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:53.058879 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 445/887] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:53.062268 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda89f6-3839-452c-a01e-163ff7392524 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:53.071183 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 443/888] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:53.071183 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d2475-5458-46ee-aec1-ab185987b83c None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:53.071183 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d2475-5458-46ee-aec1-ab185987b83c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:53.071183 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d2475-5458-46ee-aec1-ab185987b83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:53.081394 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:53.094162 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-1b8d2475-5458-46ee-aec1-ab185987b83c None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:53.095673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1b8d2475-5458-46ee-aec1-ab185987b83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:53.143224 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:53.163272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 444/889] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:53 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:15:53.164120 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d2475-5458-46ee-aec1-ab185987b83c None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:53.171437 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda89f6-3839-452c-a01e-163ff7392524 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:53.206231 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb02875-889a-4951-90f8-cc9ee5a2b1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:53.206920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb02875-889a-4951-90f8-cc9ee5a2b1d1 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:53.207205 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb02875-889a-4951-90f8-cc9ee5a2b1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:53.207480 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb02875-889a-4951-90f8-cc9ee5a2b1d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:53.208264 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb02875-889a-4951-90f8-cc9ee5a2b1d1 None admin] RBAC: Adding query filter params (interface=public, service_id=8e63d760a54b4e8b993dbc638c7856c8, region_id=RegionOne) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:53.208573 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb02875-889a-4951-90f8-cc9ee5a2b1d1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:53.209930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb02875-889a-4951-90f8-cc9ee5a2b1d1 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:53.215601 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 446/890] 10.222.0.24 () {56 vars in 1251 bytes} [Fri May 27 03:15:53 2022] GET /identity/v3/endpoints?service_id=8e63d760a54b4e8b993dbc638c7856c8&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:53.217075 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ff6ad-ea8f-42e6-9184-ab90159ddb57 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:53.217360 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ff6ad-ea8f-42e6-9184-ab90159ddb57 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:53.217631 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ff6ad-ea8f-42e6-9184-ab90159ddb57 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:53.219000 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 447/891] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:53.219571 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ff6ad-ea8f-42e6-9184-ab90159ddb57 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:53.266436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d2475-5458-46ee-aec1-ab185987b83c None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:53.293702 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:53.300486 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09226a4e-e94f-4030-b22e-47fae4d95dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:53.301296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09226a4e-e94f-4030-b22e-47fae4d95dcd None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:53.301579 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09226a4e-e94f-4030-b22e-47fae4d95dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:53.301853 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09226a4e-e94f-4030-b22e-47fae4d95dcd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:53.308778 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09226a4e-e94f-4030-b22e-47fae4d95dcd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:53.310396 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09226a4e-e94f-4030-b22e-47fae4d95dcd None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:53.318269 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ff6ad-ea8f-42e6-9184-ab90159ddb57 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:53.325356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-09226a4e-e94f-4030-b22e-47fae4d95dcd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:15:53.325356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:53.325356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:53.325356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:53.325356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:53.325356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:53.325356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:53.325356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:53.325356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:53.325356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:53.325356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:53.325356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:53.325356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:15:53.325356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:15:53.325356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:15:53.325356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:15:53.326339 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:53.326339 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:53.326339 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:15:53.326339 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:53.326339 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:15:53.326339 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:53.326339 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:53.326339 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:53.326339 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:53.326339 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:53.326339 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:53.326339 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:53.326339 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:53.326339 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:53.326339 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:53.326339 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:53.326339 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:53.326339 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:53.327370 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:53.327370 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:53.327370 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:15:53.327370 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:15:53.327370 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:15:53.327370 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:15:53.327370 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:15:53.327370 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:15:53.327370 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:15:53.327370 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:53.327370 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 445/892] 10.222.0.24 () {56 vars in 1117 bytes} [Fri May 27 03:15:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:53.327975 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09226a4e-e94f-4030-b22e-47fae4d95dcd None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:53.356355 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6debde-4eec-4a1b-b98a-64d12d7bcb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:53.357153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6debde-4eec-4a1b-b98a-64d12d7bcb17 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:53.357435 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6debde-4eec-4a1b-b98a-64d12d7bcb17 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:53.357708 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6debde-4eec-4a1b-b98a-64d12d7bcb17 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:53.358364 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6debde-4eec-4a1b-b98a-64d12d7bcb17 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:53.359752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6debde-4eec-4a1b-b98a-64d12d7bcb17 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:53.368087 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:53.371627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-4c6debde-4eec-4a1b-b98a-64d12d7bcb17 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:15:53.371627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:53.371627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:15:53.371627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:15:53.371627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:15:53.371627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:15:53.371627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:15:53.371627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:15:53.371627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:15:53.371627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:53.371627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:15:53.371627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:53.371627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:53.371627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:53.371627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:53.371627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:53.371627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:53.378895 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:53.378895 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:53.378895 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:53.378895 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:53.378895 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:53.378895 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:53.378895 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:15:53.378895 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:15:53.378895 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:15:53.378895 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:15:53.378895 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:53.378895 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:53.378895 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:53.378895 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:53.378895 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:53.378895 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:53.379906 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:53.379906 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:53.379906 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:53.379906 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:53.379906 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:53.379906 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:53.379906 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:53.379906 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:53.379906 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:53.379906 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:53.379906 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:15:53.379906 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:15:53.379906 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:15:53.379906 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:53.379906 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 448/893] 10.222.0.24 () {56 vars in 1126 bytes} [Fri May 27 03:15:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:53.380884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed91963-625e-4347-941f-3e447a41bb0b None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:53.380884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed91963-625e-4347-941f-3e447a41bb0b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:53.380884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed91963-625e-4347-941f-3e447a41bb0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:53.393796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-7ed91963-625e-4347-941f-3e447a41bb0b None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:53.394953 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-7ed91963-625e-4347-941f-3e447a41bb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:53.427493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09226a4e-e94f-4030-b22e-47fae4d95dcd None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:53.459444 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 449/894] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:53 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:15:53.459744 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25ba27-0c44-4c5a-a888-4e33815c8afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:53.460251 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed91963-625e-4347-941f-3e447a41bb0b None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:53.460444 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25ba27-0c44-4c5a-a888-4e33815c8afb None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:53.460749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25ba27-0c44-4c5a-a888-4e33815c8afb None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:53.461021 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25ba27-0c44-4c5a-a888-4e33815c8afb None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:53.461697 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25ba27-0c44-4c5a-a888-4e33815c8afb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:53.463307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25ba27-0c44-4c5a-a888-4e33815c8afb None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:53.478450 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 446/895] 10.222.0.24 () {56 vars in 1087 bytes} [Fri May 27 03:15:53 2022] GET /identity/v3/services => generated 1268 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:15:53.479393 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25ba27-0c44-4c5a-a888-4e33815c8afb None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:53.533823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:53.575354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed91963-625e-4347-941f-3e447a41bb0b None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:53.604283 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25ba27-0c44-4c5a-a888-4e33815c8afb None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:53.607185 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:53.612576 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993594d-7b76-42a6-b44e-9055cd5e3834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:53.613557 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993594d-7b76-42a6-b44e-9055cd5e3834 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:53.613841 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993594d-7b76-42a6-b44e-9055cd5e3834 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:53.614118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993594d-7b76-42a6-b44e-9055cd5e3834 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:53.615017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b993594d-7b76-42a6-b44e-9055cd5e3834 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:53.615330 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b993594d-7b76-42a6-b44e-9055cd5e3834 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:53.616816 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b993594d-7b76-42a6-b44e-9055cd5e3834 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:53.636470 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 450/896] 10.222.0.24 () {56 vars in 1149 bytes} [Fri May 27 03:15:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:53.637646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993594d-7b76-42a6-b44e-9055cd5e3834 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:53.649197 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9424c-aed3-4ccf-aa36-b44d7da1717e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:53.650192 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9424c-aed3-4ccf-aa36-b44d7da1717e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:53.650670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9424c-aed3-4ccf-aa36-b44d7da1717e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:53.651154 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9424c-aed3-4ccf-aa36-b44d7da1717e None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:53.652095 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad9424c-aed3-4ccf-aa36-b44d7da1717e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:53.652573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad9424c-aed3-4ccf-aa36-b44d7da1717e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:53.654134 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad9424c-aed3-4ccf-aa36-b44d7da1717e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:53.664587 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 447/897] 10.222.0.24 () {56 vars in 1122 bytes} [Fri May 27 03:15:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:53.665001 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9424c-aed3-4ccf-aa36-b44d7da1717e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:53.745903 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993594d-7b76-42a6-b44e-9055cd5e3834 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:53.772476 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:53.776861 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9424c-aed3-4ccf-aa36-b44d7da1717e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:53.782274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2dabc-c5f2-449b-afad-894355fdc2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:53.783133 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2dabc-c5f2-449b-afad-894355fdc2c5 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:53.790194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2dabc-c5f2-449b-afad-894355fdc2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:53.790194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2dabc-c5f2-449b-afad-894355fdc2c5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:53.790194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a2dabc-c5f2-449b-afad-894355fdc2c5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:53.790194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a2dabc-c5f2-449b-afad-894355fdc2c5 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:53.798163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-b4a2dabc-c5f2-449b-afad-894355fdc2c5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:15:53.798163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:53.798163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:15:53.798163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:15:53.798163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:15:53.798163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:15:53.798163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:15:53.798163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:15:53.798163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:15:53.798163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:53.798163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:15:53.798163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:53.798163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:53.798163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:53.798163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:53.798163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:53.799179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:53.799179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:53.799179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:53.799179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:53.799179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:53.799179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:53.799179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:53.799179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:15:53.799179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:15:53.799179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:15:53.799179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:15:53.799179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:53.799179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:53.799179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:53.799179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:53.799179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:53.800018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:53.800018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:53.800018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:53.800018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:53.800018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:53.800018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:53.800018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:53.800018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:53.800018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:53.800018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:53.800018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:53.800018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:15:53.800018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:15:53.800018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:15:53.800018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:53.800018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 451/898] 10.222.0.24 () {56 vars in 1132 bytes} [Fri May 27 03:15:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:53.801451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2dabc-c5f2-449b-afad-894355fdc2c5 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:53.819103 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb2565-0ca2-4047-8665-164e9aef053c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:53.819838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb2565-0ca2-4047-8665-164e9aef053c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:53.820061 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb2565-0ca2-4047-8665-164e9aef053c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:53.820345 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb2565-0ca2-4047-8665-164e9aef053c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:53.857821 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:53.891730 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb2565-0ca2-4047-8665-164e9aef053c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:53.893382 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb2565-0ca2-4047-8665-164e9aef053c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:53.909890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2dabc-c5f2-449b-afad-894355fdc2c5 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:53.952958 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647cadb-bbc8-4cba-9c78-aaa49ab1489b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:53.953626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647cadb-bbc8-4cba-9c78-aaa49ab1489b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:53.953908 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647cadb-bbc8-4cba-9c78-aaa49ab1489b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:53.954182 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647cadb-bbc8-4cba-9c78-aaa49ab1489b None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:53.954915 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647cadb-bbc8-4cba-9c78-aaa49ab1489b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:53.955225 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647cadb-bbc8-4cba-9c78-aaa49ab1489b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:53.956582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647cadb-bbc8-4cba-9c78-aaa49ab1489b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:53.961890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 452/899] 10.222.0.24 () {56 vars in 1122 bytes} [Fri May 27 03:15:53 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:53.962667 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647cadb-bbc8-4cba-9c78-aaa49ab1489b None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:54.020001 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 448/900] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:15:53 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:15:54.025606 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb2565-0ca2-4047-8665-164e9aef053c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:54.056950 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:54.108535 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647cadb-bbc8-4cba-9c78-aaa49ab1489b None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:54.115222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:54.140199 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb2565-0ca2-4047-8665-164e9aef053c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:54.147070 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf96ce-d563-4605-b2bf-e1b21c9b3214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:54.148025 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf96ce-d563-4605-b2bf-e1b21c9b3214 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:54.148493 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf96ce-d563-4605-b2bf-e1b21c9b3214 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:54.148901 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf96ce-d563-4605-b2bf-e1b21c9b3214 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:54.149743 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf96ce-d563-4605-b2bf-e1b21c9b3214 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:54.150191 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf96ce-d563-4605-b2bf-e1b21c9b3214 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:54.151724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf96ce-d563-4605-b2bf-e1b21c9b3214 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:54.158243 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 453/901] 10.222.0.24 () {56 vars in 1126 bytes} [Fri May 27 03:15:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:54.165299 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf96ce-d563-4605-b2bf-e1b21c9b3214 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:54.183202 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b34e2d-5495-4a47-a224-728b506e2125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:54.184018 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b34e2d-5495-4a47-a224-728b506e2125 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:54.184306 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b34e2d-5495-4a47-a224-728b506e2125 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:54.184575 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b34e2d-5495-4a47-a224-728b506e2125 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:54.197048 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b34e2d-5495-4a47-a224-728b506e2125 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:54.199224 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b34e2d-5495-4a47-a224-728b506e2125 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:54.221972 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-97b34e2d-5495-4a47-a224-728b506e2125 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:54.221972 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:54.221972 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:54.221972 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:54.221972 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:54.221972 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:54.221972 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:54.221972 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:54.221972 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:54.221972 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:54.221972 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:54.221972 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:54.221972 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:54.221972 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:54.221972 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:54.222940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:54.222940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:54.222940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:54.222940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:54.222940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:54.222940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:54.222940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:54.222940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:54.222940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:54.222940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:54.222940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:54.222940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:54.222940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:54.222940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:54.222940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:54.222940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:54.222940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:54.223828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:54.223828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:54.223828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:54.223828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:54.231169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 449/902] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:54.232210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b34e2d-5495-4a47-a224-728b506e2125 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:54.306643 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf96ce-d563-4605-b2bf-e1b21c9b3214 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:54.327496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:54.345213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6346005-b0bd-41f6-a853-d0736a5e1674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:54.346205 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6346005-b0bd-41f6-a853-d0736a5e1674 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:54.346738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6346005-b0bd-41f6-a853-d0736a5e1674 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:54.347197 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6346005-b0bd-41f6-a853-d0736a5e1674 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:54.348055 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6346005-b0bd-41f6-a853-d0736a5e1674 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:54.356560 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6346005-b0bd-41f6-a853-d0736a5e1674 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:54.356560 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6346005-b0bd-41f6-a853-d0736a5e1674 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:54.356560 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 454/903] 10.222.0.24 () {56 vars in 1126 bytes} [Fri May 27 03:15:54 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:54.361950 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6346005-b0bd-41f6-a853-d0736a5e1674 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:54.368960 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b34e2d-5495-4a47-a224-728b506e2125 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:54.391125 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:54.406638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c949a4-501b-42d5-90d4-ae43e3df5ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:54.407467 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c949a4-501b-42d5-90d4-ae43e3df5ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:54.407758 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c949a4-501b-42d5-90d4-ae43e3df5ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:54.408086 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c949a4-501b-42d5-90d4-ae43e3df5ea7 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:54.408843 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c949a4-501b-42d5-90d4-ae43e3df5ea7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:54.409178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c949a4-501b-42d5-90d4-ae43e3df5ea7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:54.410558 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c949a4-501b-42d5-90d4-ae43e3df5ea7 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:54.429049 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 450/904] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:54.430580 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c949a4-501b-42d5-90d4-ae43e3df5ea7 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:54.473274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6346005-b0bd-41f6-a853-d0736a5e1674 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:54.507362 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29974210-5641-4686-bbc9-f58b09d345b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:54.508100 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29974210-5641-4686-bbc9-f58b09d345b9 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:54.508466 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29974210-5641-4686-bbc9-f58b09d345b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:54.508739 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29974210-5641-4686-bbc9-f58b09d345b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:54.509569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29974210-5641-4686-bbc9-f58b09d345b9 None admin] RBAC: Adding query filter params (interface=public, service_id=c65f1b8f3ffb41d5b2df096dc802a58b, region_id=RegionOne) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:54.509879 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29974210-5641-4686-bbc9-f58b09d345b9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:54.511258 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29974210-5641-4686-bbc9-f58b09d345b9 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:54.516804 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 455/905] 10.222.0.24 () {56 vars in 1251 bytes} [Fri May 27 03:15:54 2022] GET /identity/v3/endpoints?service_id=c65f1b8f3ffb41d5b2df096dc802a58b&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:54.521790 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29974210-5641-4686-bbc9-f58b09d345b9 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:54.549463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c949a4-501b-42d5-90d4-ae43e3df5ea7 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:54.587279 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:54.596060 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1553a-5280-4713-b411-188641f5ad0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:54.597144 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1553a-5280-4713-b411-188641f5ad0d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:54.597907 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1553a-5280-4713-b411-188641f5ad0d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:54.598127 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1553a-5280-4713-b411-188641f5ad0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:54.610477 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa1553a-5280-4713-b411-188641f5ad0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:54.612549 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa1553a-5280-4713-b411-188641f5ad0d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:54.630412 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-faa1553a-5280-4713-b411-188641f5ad0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:54.630412 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:54.630412 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:54.630412 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:54.630412 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:54.630412 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:54.630412 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:54.630412 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:54.630412 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:54.630412 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:54.630412 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:54.630412 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:54.630412 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:15:54.630412 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:15:54.630412 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:15:54.631577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:15:54.631577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:54.631577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:54.631577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:54.631577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:54.631577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:54.631577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:54.631577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:54.631577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:54.631577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:54.631577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:54.631577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:54.631577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:54.631577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:54.631577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:54.631577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:54.631577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:54.632708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:15:54.632708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:15:54.632708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:15:54.632708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:15:54.632708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:15:54.632708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:15:54.632708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:54.632708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:54.639272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 451/906] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:15:54.641079 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1553a-5280-4713-b411-188641f5ad0d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:54.658390 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29974210-5641-4686-bbc9-f58b09d345b9 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:54.686461 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:54.703392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2bf06-36aa-40cd-a344-cc5cd274f1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:54.704370 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2bf06-36aa-40cd-a344-cc5cd274f1c2 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:54.704757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2bf06-36aa-40cd-a344-cc5cd274f1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:54.705134 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2bf06-36aa-40cd-a344-cc5cd274f1c2 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:54.705923 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2bf06-36aa-40cd-a344-cc5cd274f1c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:54.707442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2bf06-36aa-40cd-a344-cc5cd274f1c2 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:54.723269 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 456/907] 10.222.0.24 () {56 vars in 1087 bytes} [Fri May 27 03:15:54 2022] GET /identity/v3/services => generated 1268 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:15:54.756116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1553a-5280-4713-b411-188641f5ad0d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:54.797328 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551dcd3c-2dbb-4dac-8d6f-993db74a1075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:54.798352 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551dcd3c-2dbb-4dac-8d6f-993db74a1075 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:54.798812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551dcd3c-2dbb-4dac-8d6f-993db74a1075 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:54.799290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551dcd3c-2dbb-4dac-8d6f-993db74a1075 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:54.800293 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551dcd3c-2dbb-4dac-8d6f-993db74a1075 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:54.800781 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551dcd3c-2dbb-4dac-8d6f-993db74a1075 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:54.802504 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551dcd3c-2dbb-4dac-8d6f-993db74a1075 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:54.816815 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 452/908] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:15:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:54.823989 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2bf06-36aa-40cd-a344-cc5cd274f1c2 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:54.935982 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2bf06-36aa-40cd-a344-cc5cd274f1c2 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:54.946953 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:54.970823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f77597-7eaf-41f8-ba74-82ac42742792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:54.971518 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f77597-7eaf-41f8-ba74-82ac42742792 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:54.971801 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f77597-7eaf-41f8-ba74-82ac42742792 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:54.972076 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f77597-7eaf-41f8-ba74-82ac42742792 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:54.972903 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f77597-7eaf-41f8-ba74-82ac42742792 None admin] RBAC: Adding query filter params (role.id=1705f116742c4217bb8f6eab32b40850, scope.project.id=6ceffdd955324aec84ba048d2137ae00, user.id=07c9688ca035484ba5341859574c2799) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:54.973207 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f77597-7eaf-41f8-ba74-82ac42742792 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:54.974622 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f77597-7eaf-41f8-ba74-82ac42742792 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:54.981039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 457/909] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:15:54 2022] GET /identity/v3/role_assignments?user.id=07c9688ca035484ba5341859574c2799&scope.project.id=6ceffdd955324aec84ba048d2137ae00&role.id=1705f116742c4217bb8f6eab32b40850 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:55.772655 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17fad9c-38e7-4505-bb0c-fc503427369c None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:55.773256 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17fad9c-38e7-4505-bb0c-fc503427369c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:55.773256 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17fad9c-38e7-4505-bb0c-fc503427369c None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:55.774795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 453/910] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:55.780157 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b4460-8dda-40b0-8ccb-0f8cc63c1279 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:55.780567 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b4460-8dda-40b0-8ccb-0f8cc63c1279 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:55.780737 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b4460-8dda-40b0-8ccb-0f8cc63c1279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:55.798463 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-706b4460-8dda-40b0-8ccb-0f8cc63c1279 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:55.799676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-706b4460-8dda-40b0-8ccb-0f8cc63c1279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:55.868950 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 458/911] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:55 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:15:55.890784 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17fad9c-38e7-4505-bb0c-fc503427369c None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:55.996449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:56.000736 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17fad9c-38e7-4505-bb0c-fc503427369c None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:56.039922 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c80c8e-fdbc-4c2c-a5ba-5b0bc2712494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:56.040807 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c80c8e-fdbc-4c2c-a5ba-5b0bc2712494 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:56.041086 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c80c8e-fdbc-4c2c-a5ba-5b0bc2712494 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:56.041410 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c80c8e-fdbc-4c2c-a5ba-5b0bc2712494 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:56.042071 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c80c8e-fdbc-4c2c-a5ba-5b0bc2712494 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:56.043521 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c80c8e-fdbc-4c2c-a5ba-5b0bc2712494 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:56.057053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-c3c80c8e-fdbc-4c2c-a5ba-5b0bc2712494 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:15:56.057053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:56.057053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:15:56.057053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:15:56.057053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:15:56.057053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:15:56.057053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:15:56.057053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:15:56.057053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:15:56.057053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:56.057053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:15:56.057053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:56.057053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:56.057053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:56.057053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:56.057053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:56.057053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:56.058109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:56.058109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:56.058109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:56.058109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:56.058109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:56.058109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:56.058109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:15:56.058109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:15:56.058109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:15:56.058109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:15:56.058109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:56.058109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:56.058109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:56.058109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:56.058109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:56.058109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:56.058965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:56.058965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:56.058965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:56.058965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:56.058965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:56.058965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:56.058965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:56.058965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:56.058965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:56.058965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:56.058965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:15:56.058965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:15:56.058965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:15:56.058965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:56.063176 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 454/912] 10.222.0.24 () {56 vars in 1114 bytes} [Fri May 27 03:15:55 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:56.064717 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b4460-8dda-40b0-8ccb-0f8cc63c1279 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:56.175646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b4460-8dda-40b0-8ccb-0f8cc63c1279 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:56.178056 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:56.213045 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916bbd3-b8c7-4733-b5e7-3bd923a9aac5 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:56.213384 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916bbd3-b8c7-4733-b5e7-3bd923a9aac5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:56.213602 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916bbd3-b8c7-4733-b5e7-3bd923a9aac5 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:56.215728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676e481-f168-4232-b574-4f7fa555c017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:56.216287 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 455/913] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:15:56.216409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676e481-f168-4232-b574-4f7fa555c017 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:56.216674 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676e481-f168-4232-b574-4f7fa555c017 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:56.216951 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3676e481-f168-4232-b574-4f7fa555c017 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:56.217665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676e481-f168-4232-b574-4f7fa555c017 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:56.217972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676e481-f168-4232-b574-4f7fa555c017 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:56.219381 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3676e481-f168-4232-b574-4f7fa555c017 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:56.220594 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f39d0-7302-48bb-b98f-a81f03f35aeb None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:56.220887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f39d0-7302-48bb-b98f-a81f03f35aeb None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:56.221165 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f39d0-7302-48bb-b98f-a81f03f35aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:56.225018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 459/914] 10.222.0.24 () {56 vars in 1114 bytes} [Fri May 27 03:15:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:56.231798 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676e481-f168-4232-b574-4f7fa555c017 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:56.239238 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-369f39d0-7302-48bb-b98f-a81f03f35aeb None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:56.240430 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-369f39d0-7302-48bb-b98f-a81f03f35aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:56.323034 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 456/915] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:56 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:15:56.338567 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f39d0-7302-48bb-b98f-a81f03f35aeb None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:56.347962 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3676e481-f168-4232-b574-4f7fa555c017 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:56.389676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278cd2ed-e79f-45b6-9e88-380e4b9c5995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:56.390457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278cd2ed-e79f-45b6-9e88-380e4b9c5995 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:56.390747 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278cd2ed-e79f-45b6-9e88-380e4b9c5995 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:56.391061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278cd2ed-e79f-45b6-9e88-380e4b9c5995 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:56.391800 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278cd2ed-e79f-45b6-9e88-380e4b9c5995 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:56.392077 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278cd2ed-e79f-45b6-9e88-380e4b9c5995 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:56.393447 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278cd2ed-e79f-45b6-9e88-380e4b9c5995 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:56.404128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 460/916] 10.222.0.24 () {56 vars in 1114 bytes} [Fri May 27 03:15:56 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:56.406110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278cd2ed-e79f-45b6-9e88-380e4b9c5995 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:56.447137 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:56.456624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:56.464676 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f39d0-7302-48bb-b98f-a81f03f35aeb None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:56.506255 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03148c7-ae3c-4732-ad79-9abfe3d9506f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:56.507325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03148c7-ae3c-4732-ad79-9abfe3d9506f None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:56.508185 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03148c7-ae3c-4732-ad79-9abfe3d9506f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:56.508462 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03148c7-ae3c-4732-ad79-9abfe3d9506f None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:56.509192 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03148c7-ae3c-4732-ad79-9abfe3d9506f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:56.511190 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03148c7-ae3c-4732-ad79-9abfe3d9506f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:56.533723 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278cd2ed-e79f-45b6-9e88-380e4b9c5995 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:56.568446 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de45e0-155b-454f-bd5c-f2cb0df67cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:56.569133 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de45e0-155b-454f-bd5c-f2cb0df67cc4 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:56.569418 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de45e0-155b-454f-bd5c-f2cb0df67cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:56.569687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de45e0-155b-454f-bd5c-f2cb0df67cc4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:56.570983 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de45e0-155b-454f-bd5c-f2cb0df67cc4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:56.572344 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de45e0-155b-454f-bd5c-f2cb0df67cc4 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:56.598738 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 457/917] 10.222.0.24 () {60 vars in 1141 bytes} [Fri May 27 03:15:56 2022] POST /identity/v3/services => generated 241 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:15:56.658099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 461/918] 10.222.0.24 () {60 vars in 1144 bytes} [Fri May 27 03:15:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:15:57.087840 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a45b84-b213-48d9-b34e-d77781f47f79 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:57.088336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a45b84-b213-48d9-b34e-d77781f47f79 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:57.088412 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a45b84-b213-48d9-b34e-d77781f47f79 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:57.089994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 458/919] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:57.095500 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77c0ad-51aa-42b7-9ad7-8a212ee6ca1e None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:57.095947 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77c0ad-51aa-42b7-9ad7-8a212ee6ca1e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:57.096035 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77c0ad-51aa-42b7-9ad7-8a212ee6ca1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:57.117365 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-db77c0ad-51aa-42b7-9ad7-8a212ee6ca1e None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:57.118933 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-db77c0ad-51aa-42b7-9ad7-8a212ee6ca1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:57.201898 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 462/920] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:15:57.217487 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca8186-604b-41c5-9457-b57e3008e839 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:57.217795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca8186-604b-41c5-9457-b57e3008e839 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:57.218048 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca8186-604b-41c5-9457-b57e3008e839 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:57.219690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 459/921] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:57.224904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b030b6c-39c8-49b8-9d4c-bbe20acfcfda None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:57.225199 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b030b6c-39c8-49b8-9d4c-bbe20acfcfda None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:57.225475 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b030b6c-39c8-49b8-9d4c-bbe20acfcfda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:57.227687 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca8186-604b-41c5-9457-b57e3008e839 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:57.244007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-7b030b6c-39c8-49b8-9d4c-bbe20acfcfda None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:57.245197 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-7b030b6c-39c8-49b8-9d4c-bbe20acfcfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:57.255086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:57.319670 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 463/922] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:15:57.341064 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b030b6c-39c8-49b8-9d4c-bbe20acfcfda None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:57.355505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca8186-604b-41c5-9457-b57e3008e839 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:57.401838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983f722-bdf2-4107-8589-1f985e52d484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:57.403566 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983f722-bdf2-4107-8589-1f985e52d484 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:57.404002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983f722-bdf2-4107-8589-1f985e52d484 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:57.404666 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983f722-bdf2-4107-8589-1f985e52d484 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:57.405729 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983f722-bdf2-4107-8589-1f985e52d484 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:57.407937 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983f722-bdf2-4107-8589-1f985e52d484 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:57.424599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-5983f722-bdf2-4107-8589-1f985e52d484 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:15:57.424599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:57.424599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:15:57.424599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:15:57.424599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:15:57.424599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:15:57.424599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:15:57.424599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:15:57.424599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:15:57.424599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:57.424599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:15:57.424599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:57.424599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:57.424599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:57.424599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:57.424599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:57.425794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:57.425794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:57.425794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:57.425794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:57.425794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:57.425794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:57.425794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:57.425794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:15:57.425794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:15:57.425794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:15:57.425794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:15:57.425794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:57.425794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:57.425794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:57.425794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:57.425794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:57.426817 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:57.426817 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:57.426817 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:57.426817 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:57.426817 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:57.426817 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:57.426817 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:57.426817 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:57.426817 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:57.426817 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:57.426817 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:57.426817 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:15:57.426817 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:15:57.426817 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:15:57.426817 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:57.435415 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 460/923] 10.222.0.24 () {56 vars in 1132 bytes} [Fri May 27 03:15:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:57.437200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983f722-bdf2-4107-8589-1f985e52d484 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:57.455595 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b030b6c-39c8-49b8-9d4c-bbe20acfcfda None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:57.483220 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:57.502674 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46cb74-735e-4e36-ac07-68c46293af3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:57.503677 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46cb74-735e-4e36-ac07-68c46293af3f None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:57.504028 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46cb74-735e-4e36-ac07-68c46293af3f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:57.504392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46cb74-735e-4e36-ac07-68c46293af3f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:57.518285 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa46cb74-735e-4e36-ac07-68c46293af3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:15:57.528866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-aa46cb74-735e-4e36-ac07-68c46293af3f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:15:57.528866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:57.528866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:57.528866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:57.528866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:57.528866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:57.528866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:57.528866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:57.528866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:57.528866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:57.528866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:57.528866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:57.528866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:15:57.528866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:15:57.528866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:15:57.528866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:15:57.529796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:15:57.529796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:15:57.529796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:15:57.529796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:15:57.529796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:57.529796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:57.529796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:57.529796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:57.529796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:57.529796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:57.529796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:57.529796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:57.529796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:57.529796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:57.529796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:57.529796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:57.530611 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:57.530611 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:57.530611 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:57.530611 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:57.530611 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:15:57.530611 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:15:57.530611 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:15:57.530611 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:15:57.530611 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:15:57.530611 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:15:57.530611 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:15:57.530611 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:57.536848 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 464/924] 10.222.0.24 () {56 vars in 1120 bytes} [Fri May 27 03:15:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:15:57.536848 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46cb74-735e-4e36-ac07-68c46293af3f None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:57.540530 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:57.562658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983f722-bdf2-4107-8589-1f985e52d484 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:57.607749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab79b96-b4a5-4bda-b9db-fb78254fe01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:57.608365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab79b96-b4a5-4bda-b9db-fb78254fe01d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:57.608649 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab79b96-b4a5-4bda-b9db-fb78254fe01d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:57.608924 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab79b96-b4a5-4bda-b9db-fb78254fe01d None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:57.609786 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab79b96-b4a5-4bda-b9db-fb78254fe01d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:57.610110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab79b96-b4a5-4bda-b9db-fb78254fe01d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:57.611982 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab79b96-b4a5-4bda-b9db-fb78254fe01d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:57.622925 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 461/925] 10.222.0.24 () {56 vars in 1126 bytes} [Fri May 27 03:15:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:15:57.625618 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab79b96-b4a5-4bda-b9db-fb78254fe01d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:57.650965 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46cb74-735e-4e36-ac07-68c46293af3f None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:57.691746 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ce620-1990-45c7-8229-c6ede27e4585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:57.692410 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ce620-1990-45c7-8229-c6ede27e4585 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:57.692715 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ce620-1990-45c7-8229-c6ede27e4585 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:57.692975 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ce620-1990-45c7-8229-c6ede27e4585 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:57.693717 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ce620-1990-45c7-8229-c6ede27e4585 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:57.694021 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ce620-1990-45c7-8229-c6ede27e4585 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:57.695441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ce620-1990-45c7-8229-c6ede27e4585 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:57.709620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 465/926] 10.222.0.24 () {56 vars in 1115 bytes} [Fri May 27 03:15:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:57.711164 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ce620-1990-45c7-8229-c6ede27e4585 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:57.736478 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:57.761107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab79b96-b4a5-4bda-b9db-fb78254fe01d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:57.787646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:57.799487 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75bb7c6-50fc-427b-93cc-6da1d37b0ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:57.802500 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bb7c6-50fc-427b-93cc-6da1d37b0ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:57.802500 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bb7c6-50fc-427b-93cc-6da1d37b0ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:57.802500 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bb7c6-50fc-427b-93cc-6da1d37b0ba6 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:57.802500 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75bb7c6-50fc-427b-93cc-6da1d37b0ba6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:57.802500 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75bb7c6-50fc-427b-93cc-6da1d37b0ba6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:57.805272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75bb7c6-50fc-427b-93cc-6da1d37b0ba6 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:57.811270 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 462/927] 10.222.0.24 () {56 vars in 1126 bytes} [Fri May 27 03:15:57 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:57.819549 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75bb7c6-50fc-427b-93cc-6da1d37b0ba6 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:57.827918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ce620-1990-45c7-8229-c6ede27e4585 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:57.867797 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73fdec-bc9f-4e6b-86c1-c90597a0b085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:57.868620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73fdec-bc9f-4e6b-86c1-c90597a0b085 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:57.868984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73fdec-bc9f-4e6b-86c1-c90597a0b085 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:57.869357 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73fdec-bc9f-4e6b-86c1-c90597a0b085 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:57.881475 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af73fdec-bc9f-4e6b-86c1-c90597a0b085 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:57.883919 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af73fdec-bc9f-4e6b-86c1-c90597a0b085 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:57.902847 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-af73fdec-bc9f-4e6b-86c1-c90597a0b085 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:57.902847 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:57.902847 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:57.902847 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:57.902847 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:57.902847 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:57.902847 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:57.902847 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:57.902847 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:57.902847 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:57.902847 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:57.902847 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:57.902847 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:57.902847 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:57.902847 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:57.903774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:57.903774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:57.903774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:57.903774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:57.903774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:57.903774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:57.903774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:57.903774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:57.903774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:57.903774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:57.903774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:57.903774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:57.903774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:57.903774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:57.903774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:57.903774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:57.903774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:57.904670 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:57.904670 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:57.904670 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:57.904670 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:57.908994 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 466/928] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:57.910491 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73fdec-bc9f-4e6b-86c1-c90597a0b085 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:57.934564 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75bb7c6-50fc-427b-93cc-6da1d37b0ba6 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:57.974562 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444b676-6764-4cf2-aeed-16e613d9bf5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:57.975784 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444b676-6764-4cf2-aeed-16e613d9bf5b None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:57.976303 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444b676-6764-4cf2-aeed-16e613d9bf5b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:57.976758 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3444b676-6764-4cf2-aeed-16e613d9bf5b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:57.977980 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3444b676-6764-4cf2-aeed-16e613d9bf5b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:57.980545 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3444b676-6764-4cf2-aeed-16e613d9bf5b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:58.019427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73fdec-bc9f-4e6b-86c1-c90597a0b085 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:58.037534 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:58.053907 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40f634-2239-4978-81f1-99c5ce8791f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:58.054657 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40f634-2239-4978-81f1-99c5ce8791f4 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:58.054962 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40f634-2239-4978-81f1-99c5ce8791f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:58.055239 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf40f634-2239-4978-81f1-99c5ce8791f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:58.055954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf40f634-2239-4978-81f1-99c5ce8791f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:58.056317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf40f634-2239-4978-81f1-99c5ce8791f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:58.057682 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf40f634-2239-4978-81f1-99c5ce8791f4 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:58.069286 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 467/929] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:15:58.076553 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40f634-2239-4978-81f1-99c5ce8791f4 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:58.151847 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 463/930] 10.222.0.24 () {60 vars in 1144 bytes} [Fri May 27 03:15:57 2022] POST /identity/v3/endpoints => generated 347 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:15:58.190338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf40f634-2239-4978-81f1-99c5ce8791f4 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:58.239893 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dedcd0-f2c2-4349-ab64-5751a7785a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:58.241178 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dedcd0-f2c2-4349-ab64-5751a7785a71 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:58.242240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dedcd0-f2c2-4349-ab64-5751a7785a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:58.242747 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dedcd0-f2c2-4349-ab64-5751a7785a71 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:58.251888 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dedcd0-f2c2-4349-ab64-5751a7785a71 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:58.253893 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dedcd0-f2c2-4349-ab64-5751a7785a71 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:58.272201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-52dedcd0-f2c2-4349-ab64-5751a7785a71 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:15:58.272201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:58.272201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:58.272201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:58.272201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:58.272201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:58.272201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:58.272201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:58.272201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:58.272201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:58.272201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:58.272201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:58.272201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:15:58.272201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:15:58.272201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:15:58.272201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:15:58.273272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:58.273272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:58.273272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:15:58.273272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:58.273272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:15:58.273272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:15:58.273272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:58.273272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:58.273272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:58.273272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:58.273272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:58.273272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:58.273272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:58.273272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:58.273272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:58.273272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:58.273272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:58.273272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:58.274244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:58.274244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:58.274244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:15:58.274244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:15:58.274244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:15:58.274244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:15:58.274244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:15:58.274244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:15:58.274244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:15:58.274244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:58.277150 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 468/931] 10.222.0.24 () {56 vars in 1117 bytes} [Fri May 27 03:15:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:15:58.283345 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444b676-6764-4cf2-aeed-16e613d9bf5b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:58.285792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:58.395181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3444b676-6764-4cf2-aeed-16e613d9bf5b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:58.431441 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012cd159-30e4-4076-ae37-e06edbba34b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:58.432482 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012cd159-30e4-4076-ae37-e06edbba34b0 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:58.432966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012cd159-30e4-4076-ae37-e06edbba34b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:58.433450 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012cd159-30e4-4076-ae37-e06edbba34b0 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:58.434501 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012cd159-30e4-4076-ae37-e06edbba34b0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:58.435055 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012cd159-30e4-4076-ae37-e06edbba34b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:58.436707 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012cd159-30e4-4076-ae37-e06edbba34b0 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:58.460465 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 464/932] 10.222.0.24 () {56 vars in 1149 bytes} [Fri May 27 03:15:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:58.462234 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dedcd0-f2c2-4349-ab64-5751a7785a71 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:58.498090 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:58.570641 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dedcd0-f2c2-4349-ab64-5751a7785a71 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:58.607340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3f291-6a60-4f68-a637-3c59b9bd8d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:58.608139 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3f291-6a60-4f68-a637-3c59b9bd8d39 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:58.608414 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3f291-6a60-4f68-a637-3c59b9bd8d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:58.608746 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3f291-6a60-4f68-a637-3c59b9bd8d39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:58.619831 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3f291-6a60-4f68-a637-3c59b9bd8d39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:58.621268 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3f291-6a60-4f68-a637-3c59b9bd8d39 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:58.645909 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-e5d3f291-6a60-4f68-a637-3c59b9bd8d39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:58.645909 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:58.645909 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:58.645909 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:58.645909 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:58.645909 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:58.645909 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:58.645909 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:58.645909 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:58.645909 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:58.645909 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:58.645909 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:58.645909 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:15:58.645909 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:15:58.645909 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:15:58.647971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:15:58.647971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:58.647971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:58.647971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:58.647971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:58.647971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:58.647971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:58.647971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:58.647971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:58.647971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:58.647971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:58.647971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:58.647971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:58.647971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:58.647971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:58.647971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:58.647971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:58.649082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:15:58.649082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:15:58.649082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:15:58.649082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:58.649082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 469/933] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:15:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:58.655538 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012cd159-30e4-4076-ae37-e06edbba34b0 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:58.728238 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:58.773968 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012cd159-30e4-4076-ae37-e06edbba34b0 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:58.810227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1d9c2-bccf-4cb8-8f66-84e132d47c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:58.811365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1d9c2-bccf-4cb8-8f66-84e132d47c45 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:58.811858 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1d9c2-bccf-4cb8-8f66-84e132d47c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:58.812355 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1d9c2-bccf-4cb8-8f66-84e132d47c45 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:58.813328 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1d9c2-bccf-4cb8-8f66-84e132d47c45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:58.813866 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1d9c2-bccf-4cb8-8f66-84e132d47c45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:58.815527 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1d9c2-bccf-4cb8-8f66-84e132d47c45 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:58.830454 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 465/934] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:58.836717 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3f291-6a60-4f68-a637-3c59b9bd8d39 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:58.879949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1a499-577b-43f8-8765-66b1f8cbaee0 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:58.880573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1a499-577b-43f8-8765-66b1f8cbaee0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:58.881341 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1a499-577b-43f8-8765-66b1f8cbaee0 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:58.884155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 466/935] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:58.888741 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08e3f8-fc90-4140-9c78-4eeb1f904398 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:58.889030 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08e3f8-fc90-4140-9c78-4eeb1f904398 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:58.889312 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08e3f8-fc90-4140-9c78-4eeb1f904398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:58.906709 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-6d08e3f8-fc90-4140-9c78-4eeb1f904398 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:58.907928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6d08e3f8-fc90-4140-9c78-4eeb1f904398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:58.940984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3f291-6a60-4f68-a637-3c59b9bd8d39 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:58.975229 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:58.979627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05030f-5b57-4adf-8e0f-5c7df626d231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:58.980527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05030f-5b57-4adf-8e0f-5c7df626d231 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:58.980811 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05030f-5b57-4adf-8e0f-5c7df626d231 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:58.981089 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05030f-5b57-4adf-8e0f-5c7df626d231 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:58.983526 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 467/936] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:15:58.992628 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d05030f-5b57-4adf-8e0f-5c7df626d231 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:58.994181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d05030f-5b57-4adf-8e0f-5c7df626d231 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:59.007154 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d08e3f8-fc90-4140-9c78-4eeb1f904398 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:59.009745 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-5d05030f-5b57-4adf-8e0f-5c7df626d231 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:59.009745 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:59.009745 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:59.009745 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:59.009745 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:59.009745 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:59.009745 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:59.009745 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:59.009745 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:59.009745 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:59.009745 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:59.009745 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:59.009745 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:15:59.009745 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:15:59.009745 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:15:59.010625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:15:59.010625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:59.010625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:59.010625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:59.010625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:59.010625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:59.010625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:59.010625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:59.010625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:15:59.010625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:59.010625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:59.010625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:59.010625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:59.010625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:59.010625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:59.010625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:59.010625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:59.011870 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:15:59.011870 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:15:59.011870 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:15:59.011870 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:15:59.011870 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:15:59.011870 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:15:59.011870 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:59.011870 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:15:59.012904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 470/937] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:59.018901 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05030f-5b57-4adf-8e0f-5c7df626d231 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:59.064023 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:59.115504 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d08e3f8-fc90-4140-9c78-4eeb1f904398 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:59.144130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05030f-5b57-4adf-8e0f-5c7df626d231 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:59.152629 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6eca1b-badd-403f-835a-fb428473e191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:59.153434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6eca1b-badd-403f-835a-fb428473e191 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:59.153722 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6eca1b-badd-403f-835a-fb428473e191 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:59.153999 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6eca1b-badd-403f-835a-fb428473e191 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:59.154650 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6eca1b-badd-403f-835a-fb428473e191 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:59.156222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6eca1b-badd-403f-835a-fb428473e191 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:59.168644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-3b6eca1b-badd-403f-835a-fb428473e191 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:15:59.168644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:59.168644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:15:59.168644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:15:59.168644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:15:59.168644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:15:59.168644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:15:59.168644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:15:59.168644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:15:59.168644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:59.168644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:15:59.168644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:59.168644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:59.168644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:59.168644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:59.168644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:59.168644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:59.169830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:59.169830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:59.169830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:59.169830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:59.169830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:59.169830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:59.169830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:15:59.169830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:15:59.169830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:15:59.169830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:15:59.169830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:59.169830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:59.169830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:59.169830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:59.169830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:59.169830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:59.170679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:59.170679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:59.170679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:59.170679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:59.170679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:59.170679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:59.170679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:59.170679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:59.170679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:59.170679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:59.170679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:15:59.170679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:15:59.170679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:15:59.170679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:59.170679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 468/938] 10.222.0.24 () {56 vars in 1126 bytes} [Fri May 27 03:15:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:15:59.171642 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30efc2-3ccd-459c-8a5a-335edcf4938b None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:59.171642 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30efc2-3ccd-459c-8a5a-335edcf4938b None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:59.171883 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30efc2-3ccd-459c-8a5a-335edcf4938b None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:59.173286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 469/939] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:15:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:59.177838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30efc2-3ccd-459c-8a5a-335edcf4938b None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:59.187287 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a944810-06cf-4ae6-93d1-6414af6ffef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:59.188047 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a944810-06cf-4ae6-93d1-6414af6ffef6 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:59.188356 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a944810-06cf-4ae6-93d1-6414af6ffef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:59.188633 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a944810-06cf-4ae6-93d1-6414af6ffef6 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:59.189425 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a944810-06cf-4ae6-93d1-6414af6ffef6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:59.189740 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a944810-06cf-4ae6-93d1-6414af6ffef6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:59.191197 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a944810-06cf-4ae6-93d1-6414af6ffef6 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:59.209515 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 471/940] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:15:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:59.209515 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bff967-1896-4ae5-9cea-2ea47cc0e33d None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:59.209515 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bff967-1896-4ae5-9cea-2ea47cc0e33d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:59.209515 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bff967-1896-4ae5-9cea-2ea47cc0e33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:59.218049 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:59.223811 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-20bff967-1896-4ae5-9cea-2ea47cc0e33d None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:59.225006 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-20bff967-1896-4ae5-9cea-2ea47cc0e33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:59.302870 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 472/941] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:15:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:15:59.302870 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bff967-1896-4ae5-9cea-2ea47cc0e33d None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:59.304939 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30efc2-3ccd-459c-8a5a-335edcf4938b None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:59.324341 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:59.346093 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08086889-19af-420c-afb6-a77b5f5ed5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:59.348697 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08086889-19af-420c-afb6-a77b5f5ed5fd None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:59.348697 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08086889-19af-420c-afb6-a77b5f5ed5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:59.348697 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08086889-19af-420c-afb6-a77b5f5ed5fd None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:59.348697 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08086889-19af-420c-afb6-a77b5f5ed5fd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:59.348697 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08086889-19af-420c-afb6-a77b5f5ed5fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:59.350301 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08086889-19af-420c-afb6-a77b5f5ed5fd None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:59.360178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 470/942] 10.222.0.24 () {56 vars in 1122 bytes} [Fri May 27 03:15:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:59.360178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08086889-19af-420c-afb6-a77b5f5ed5fd None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:59.409517 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bff967-1896-4ae5-9cea-2ea47cc0e33d None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:59.448853 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa1f12-7fb8-49b5-9870-10f56720d619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:59.450442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa1f12-7fb8-49b5-9870-10f56720d619 None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:59.450726 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa1f12-7fb8-49b5-9870-10f56720d619 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:59.451037 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa1f12-7fb8-49b5-9870-10f56720d619 None admin] PATH_INFO: `/v3/projects/6ceffdd955324aec84ba048d2137ae00/users/07c9688ca035484ba5341859574c2799/roles/1705f116742c4217bb8f6eab32b40850` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:59.454191 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa1f12-7fb8-49b5-9870-10f56720d619 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ceffdd955324aec84ba048d2137ae00, user_id=07c9688ca035484ba5341859574c2799, role_id=1705f116742c4217bb8f6eab32b40850)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:59.455951 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aa1f12-7fb8-49b5-9870-10f56720d619 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:59.460231 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:59.462444 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08086889-19af-420c-afb6-a77b5f5ed5fd None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:59.497268 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1872b2-1cb1-4e12-9d44-a8d6b87e7f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:59.497922 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1872b2-1cb1-4e12-9d44-a8d6b87e7f6f None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:59.498199 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1872b2-1cb1-4e12-9d44-a8d6b87e7f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:59.498468 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1872b2-1cb1-4e12-9d44-a8d6b87e7f6f None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:59.499197 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1872b2-1cb1-4e12-9d44-a8d6b87e7f6f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:59.500626 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1872b2-1cb1-4e12-9d44-a8d6b87e7f6f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:59.501966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: WARNING keystone.server.flask.common [None req-1a1872b2-1cb1-4e12-9d44-a8d6b87e7f6f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 27 03:15:59.532827 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 473/943] 10.222.0.24 () {58 vars in 1439 bytes} [Fri May 27 03:15:59 2022] PUT /identity/v3/projects/6ceffdd955324aec84ba048d2137ae00/users/07c9688ca035484ba5341859574c2799/roles/1705f116742c4217bb8f6eab32b40850 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:15:59.533648 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa1f12-7fb8-49b5-9870-10f56720d619 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:59.547658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.sql.core [None req-1a1872b2-1cb1-4e12-9d44-a8d6b87e7f6f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 27 03:15:59.547658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 27 03:15:59.547658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [parameters: {'id': '90bef3d88d9c40d09e009224d3a97701', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 27 03:15:59.547658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80482) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 27 03:15:59.556979 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-1a1872b2-1cb1-4e12-9d44-a8d6b87e7f6f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 90bef3d88d9c40d09e009224d3a97701.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 90bef3d88d9c40d09e009224d3a97701. May 27 03:15:59.556979 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:59.556979 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 03:15:59.556979 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 03:15:59.556979 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 03:15:59.556979 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 03:15:59.556979 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 03:15:59.556979 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application result = self._query(query) May 27 03:15:59.556979 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 03:15:59.556979 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application conn.query(q) May 27 03:15:59.556979 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 03:15:59.556979 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 03:15:59.556979 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 03:15:59.557970 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application result.read() May 27 03:15:59.557970 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 03:15:59.557970 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 03:15:59.557970 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 03:15:59.557970 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 03:15:59.557970 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 03:15:59.557970 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 03:15:59.557970 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 03:15:59.557970 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 03:15:59.557970 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 27 03:15:59.557970 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:59.557970 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 27 03:15:59.557970 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:59.557970 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:59.557970 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 27 03:15:59.557970 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 27 03:15:59.557970 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 27 03:15:59.557970 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return project_ref.to_dict() May 27 03:15:59.559000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 03:15:59.559000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application next(self.gen) May 27 03:15:59.559000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 27 03:15:59.559000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application yield resource May 27 03:15:59.559000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 03:15:59.559000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application next(self.gen) May 27 03:15:59.559000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 27 03:15:59.559000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self.session.rollback() May 27 03:15:59.559000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 27 03:15:59.559000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self.force_reraise() May 27 03:15:59.559000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 27 03:15:59.559000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise self.value May 27 03:15:59.559000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 27 03:15:59.559000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 27 03:15:59.559000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 27 03:15:59.559000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application session.commit() May 27 03:15:59.559000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 27 03:15:59.560466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 27 03:15:59.560466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 27 03:15:59.560466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._prepare_impl() May 27 03:15:59.560466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 27 03:15:59.560466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self.session.flush() May 27 03:15:59.560466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 27 03:15:59.560466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._flush(objects) May 27 03:15:59.560466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 27 03:15:59.560466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 27 03:15:59.560466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 27 03:15:59.560466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application compat.raise_( May 27 03:15:59.560466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 03:15:59.560466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception May 27 03:15:59.560466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 27 03:15:59.560466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application flush_context.execute() May 27 03:15:59.560466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 27 03:15:59.560466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rec.execute(self) May 27 03:15:59.561578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 27 03:15:59.561578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 27 03:15:59.561578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 27 03:15:59.561578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application _emit_insert_statements( May 27 03:15:59.561578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 27 03:15:59.561578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application c = connection._execute_20( May 27 03:15:59.561578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 27 03:15:59.561578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 27 03:15:59.561578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 27 03:15:59.561578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 27 03:15:59.561578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 27 03:15:59.561578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ret = self._execute_context( May 27 03:15:59.561578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 27 03:15:59.561578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 27 03:15:59.561578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 27 03:15:59.562520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 27 03:15:59.562520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 03:15:59.562520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception May 27 03:15:59.562520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 03:15:59.562520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 03:15:59.562520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 03:15:59.562520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 03:15:59.562520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 03:15:59.562520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application result = self._query(query) May 27 03:15:59.562520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 03:15:59.562520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application conn.query(q) May 27 03:15:59.562520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 03:15:59.562520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 03:15:59.562520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 03:15:59.562520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application result.read() May 27 03:15:59.562520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 03:15:59.562520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 03:15:59.564024 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 03:15:59.564024 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 03:15:59.564024 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 03:15:59.564024 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 03:15:59.564024 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 03:15:59.564024 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 03:15:59.564024 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 27 03:15:59.564024 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 27 03:15:59.564024 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [parameters: {'id': '90bef3d88d9c40d09e009224d3a97701', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 27 03:15:59.564024 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 27 03:15:59.564024 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:59.564024 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:15:59.564024 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:59.564024 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:59.564024 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 27 03:15:59.565040 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 27 03:15:59.565040 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 27 03:15:59.565040 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 27 03:15:59.565040 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 27 03:15:59.565040 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:59.565040 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:15:59.565040 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:59.565040 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:59.565040 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:59.565040 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:59.565040 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:59.565040 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:59.565040 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:59.565040 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:59.565040 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:59.565040 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:59.565040 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:59.566520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:59.566520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 27 03:15:59.566520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 27 03:15:59.566520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:59.566520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:59.566520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 27 03:15:59.566520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.Conflict( May 27 03:15:59.566520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 90bef3d88d9c40d09e009224d3a97701. May 27 03:15:59.566520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:59.566520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 471/944] 10.222.0.24 () {60 vars in 1140 bytes} [Fri May 27 03:15:59 2022] POST /identity/v3/projects => generated 262 bytes in 203 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 27 03:15:59.567897 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1872b2-1cb1-4e12-9d44-a8d6b87e7f6f None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:59.611460 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:59.677044 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aa1f12-7fb8-49b5-9870-10f56720d619 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:59.683488 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1872b2-1cb1-4e12-9d44-a8d6b87e7f6f None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:59.719185 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8224a61-4378-48c7-947f-dcaeb0f7fc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:59.720024 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8224a61-4378-48c7-947f-dcaeb0f7fc8d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:59.720398 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8224a61-4378-48c7-947f-dcaeb0f7fc8d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:59.720771 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8224a61-4378-48c7-947f-dcaeb0f7fc8d None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:59.721586 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8224a61-4378-48c7-947f-dcaeb0f7fc8d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:59.721993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8224a61-4378-48c7-947f-dcaeb0f7fc8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:59.722515 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e5c59-0ac1-42ce-9de9-508c0d39d4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:59.723566 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e5c59-0ac1-42ce-9de9-508c0d39d4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:59.723697 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8224a61-4378-48c7-947f-dcaeb0f7fc8d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:59.724813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e5c59-0ac1-42ce-9de9-508c0d39d4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:59.725262 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e5c59-0ac1-42ce-9de9-508c0d39d4c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:59.730263 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 474/945] 10.222.0.24 () {56 vars in 1122 bytes} [Fri May 27 03:15:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:59.736393 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e5c59-0ac1-42ce-9de9-508c0d39d4c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:59.736549 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8224a61-4378-48c7-947f-dcaeb0f7fc8d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:59.737288 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e5c59-0ac1-42ce-9de9-508c0d39d4c5 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:59.759120 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-e42e5c59-0ac1-42ce-9de9-508c0d39d4c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:59.759120 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:15:59.759120 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:15:59.759120 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:15:59.759120 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:15:59.759120 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:15:59.759120 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:15:59.759120 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:15:59.759120 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:15:59.759120 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:15:59.759120 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:15:59.759120 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:15:59.759120 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:15:59.759120 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:15:59.759120 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:15:59.760235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:15:59.760235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:15:59.760235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:15:59.760235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:15:59.760235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:15:59.760235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:15:59.760235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:15:59.760235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:15:59.760235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:15:59.760235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:15:59.760235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:15:59.760235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:15:59.760235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:15:59.760235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:15:59.760235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:15:59.760235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:15:59.760235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:15:59.761401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:15:59.761401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:15:59.761401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:15:59.761401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:15:59.761401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:15:59.761401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:15:59.761401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:15:59.761401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:15:59.761401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 472/946] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:15:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:15:59.766817 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e5c59-0ac1-42ce-9de9-508c0d39d4c5 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:59.813507 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:59.867671 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:59.871956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8224a61-4378-48c7-947f-dcaeb0f7fc8d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:59.874477 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e5c59-0ac1-42ce-9de9-508c0d39d4c5 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:59.909748 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a5100b-1800-4672-ba16-65c5897f7e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:59.910570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5100b-1800-4672-ba16-65c5897f7e8b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:59.910980 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5100b-1800-4672-ba16-65c5897f7e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:59.911353 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5100b-1800-4672-ba16-65c5897f7e8b None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:59.912207 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a5100b-1800-4672-ba16-65c5897f7e8b None admin] RBAC: Adding query filter params (name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:59.912630 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a5100b-1800-4672-ba16-65c5897f7e8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:59.914196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a5100b-1800-4672-ba16-65c5897f7e8b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:59.915133 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234f8a4-0467-4b0a-8c64-47ecd066e2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:59.915787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234f8a4-0467-4b0a-8c64-47ecd066e2d1 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:59.916346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234f8a4-0467-4b0a-8c64-47ecd066e2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:59.916619 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234f8a4-0467-4b0a-8c64-47ecd066e2d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:59.917409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234f8a4-0467-4b0a-8c64-47ecd066e2d1 None admin] RBAC: Adding query filter params (interface=public, service_id=541921ac6534459b9fbcf826bcd8e51d, region_id=RegionOne) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:15:59.917719 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234f8a4-0467-4b0a-8c64-47ecd066e2d1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:59.919684 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234f8a4-0467-4b0a-8c64-47ecd066e2d1 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:59.925417 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 475/947] 10.222.0.24 () {56 vars in 1251 bytes} [Fri May 27 03:15:59 2022] GET /identity/v3/endpoints?service_id=541921ac6534459b9fbcf826bcd8e51d&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:15:59.931427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234f8a4-0467-4b0a-8c64-47ecd066e2d1 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:59.942961 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 473/948] 10.222.0.24 () {56 vars in 1112 bytes} [Fri May 27 03:15:59 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:16:00.048154 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234f8a4-0467-4b0a-8c64-47ecd066e2d1 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:00.079353 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:00.093600 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b697f53b-fcfd-44cd-b3c0-33522f59e350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:00.098188 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697f53b-fcfd-44cd-b3c0-33522f59e350 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:00.098188 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697f53b-fcfd-44cd-b3c0-33522f59e350 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:00.098188 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697f53b-fcfd-44cd-b3c0-33522f59e350 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:00.098188 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b697f53b-fcfd-44cd-b3c0-33522f59e350 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:00.098188 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b697f53b-fcfd-44cd-b3c0-33522f59e350 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:00.104353 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 476/949] 10.222.0.24 () {56 vars in 1087 bytes} [Fri May 27 03:15:59 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:16:00.522546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3c57d-d496-42a1-91d4-b6f409d44de2 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:00.523114 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3c57d-d496-42a1-91d4-b6f409d44de2 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:00.523114 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3c57d-d496-42a1-91d4-b6f409d44de2 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:00.524678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 474/950] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:00.530117 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d004cef-5897-4965-8d6c-86534e795740 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:00.530527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d004cef-5897-4965-8d6c-86534e795740 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:00.530670 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d004cef-5897-4965-8d6c-86534e795740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:00.547576 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-4d004cef-5897-4965-8d6c-86534e795740 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:00.548756 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4d004cef-5897-4965-8d6c-86534e795740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:00.620063 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 477/951] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:16:00.642333 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3c57d-d496-42a1-91d4-b6f409d44de2 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:00.760323 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3c57d-d496-42a1-91d4-b6f409d44de2 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:00.797127 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0718b1-b8b2-498d-ae39-fd31aa8f1235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:00.798213 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0718b1-b8b2-498d-ae39-fd31aa8f1235 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:00.798642 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0718b1-b8b2-498d-ae39-fd31aa8f1235 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:00.799097 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0718b1-b8b2-498d-ae39-fd31aa8f1235 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:00.799897 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0718b1-b8b2-498d-ae39-fd31aa8f1235 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:00.801420 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0718b1-b8b2-498d-ae39-fd31aa8f1235 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:00.814175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-9b0718b1-b8b2-498d-ae39-fd31aa8f1235 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:16:00.814175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:00.814175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:16:00.814175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:16:00.814175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:16:00.814175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:16:00.814175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:16:00.814175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:16:00.814175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:16:00.814175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:16:00.814175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:16:00.814175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:16:00.814175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:00.814175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:00.814175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:00.814175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:00.814175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:00.815508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:00.815508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:00.815508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:00.815508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:00.815508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:00.815508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:00.815508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:16:00.815508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:16:00.815508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:16:00.815508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:16:00.815508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:00.815508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:00.815508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:00.815508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:00.815508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:00.815508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:00.816593 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:00.816593 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:16:00.816593 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:00.816593 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:00.816593 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:00.816593 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:00.816593 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:00.816593 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:00.816593 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:00.816593 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:00.816593 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:16:00.816593 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:16:00.816593 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:16:00.816593 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:16:00.819166 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 475/952] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:16:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:16:00.828003 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d004cef-5897-4965-8d6c-86534e795740 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:00.926765 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d004cef-5897-4965-8d6c-86534e795740 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:00.945902 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:00.963246 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb212ea-dbea-4e10-bc03-462a1a577ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:00.963995 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb212ea-dbea-4e10-bc03-462a1a577ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:00.964397 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb212ea-dbea-4e10-bc03-462a1a577ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:00.964744 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb212ea-dbea-4e10-bc03-462a1a577ac5 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:00.965522 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb212ea-dbea-4e10-bc03-462a1a577ac5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:00.965953 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb212ea-dbea-4e10-bc03-462a1a577ac5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:00.967481 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb212ea-dbea-4e10-bc03-462a1a577ac5 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:00.977023 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 478/953] 10.222.0.24 () {56 vars in 1112 bytes} [Fri May 27 03:16:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:00.979016 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0718b1-b8b2-498d-ae39-fd31aa8f1235 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:01.000544 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:01.088557 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0718b1-b8b2-498d-ae39-fd31aa8f1235 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:01.125034 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152400e1-dda6-495b-bdc4-00a98ca1ba2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:01.125831 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152400e1-dda6-495b-bdc4-00a98ca1ba2d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:01.126184 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152400e1-dda6-495b-bdc4-00a98ca1ba2d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:01.126557 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152400e1-dda6-495b-bdc4-00a98ca1ba2d None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:01.127355 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152400e1-dda6-495b-bdc4-00a98ca1ba2d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:01.127732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152400e1-dda6-495b-bdc4-00a98ca1ba2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:01.129182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152400e1-dda6-495b-bdc4-00a98ca1ba2d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:01.139581 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 476/954] 10.222.0.24 () {56 vars in 1112 bytes} [Fri May 27 03:16:00 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:01.866218 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf685e-f97f-48a8-a450-c22ab6c5d710 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:01.866804 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf685e-f97f-48a8-a450-c22ab6c5d710 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:01.866908 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cf685e-f97f-48a8-a450-c22ab6c5d710 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:01.868491 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 479/955] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:01.874172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc5cdc-584b-433f-a136-20b52d420878 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:01.874617 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc5cdc-584b-433f-a136-20b52d420878 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:01.874800 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc5cdc-584b-433f-a136-20b52d420878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:01.892419 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e7bc5cdc-584b-433f-a136-20b52d420878 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:01.893935 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e7bc5cdc-584b-433f-a136-20b52d420878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:01.964028 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 477/956] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:16:01.988161 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf685e-f97f-48a8-a450-c22ab6c5d710 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:02.096015 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cf685e-f97f-48a8-a450-c22ab6c5d710 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:02.131047 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7cd2f6-b613-4c30-ad52-570937c88ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:02.131785 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cd2f6-b613-4c30-ad52-570937c88ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:02.132687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cd2f6-b613-4c30-ad52-570937c88ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:02.132969 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cd2f6-b613-4c30-ad52-570937c88ab6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:02.143339 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7cd2f6-b613-4c30-ad52-570937c88ab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:16:02.151437 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-5a7cd2f6-b613-4c30-ad52-570937c88ab6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:16:02.151437 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:02.151437 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:02.151437 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:02.151437 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:02.151437 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:02.151437 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:02.151437 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:02.151437 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:02.151437 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:02.151437 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:02.151437 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:02.151437 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:16:02.151437 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:16:02.151437 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:16:02.151437 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:16:02.152554 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:16:02.152554 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:16:02.152554 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:16:02.152554 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:16:02.152554 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:02.152554 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:02.152554 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:02.152554 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:02.152554 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:02.152554 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:02.152554 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:02.152554 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:16:02.152554 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:02.152554 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:02.152554 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:02.152554 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:02.153585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:02.153585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:02.153585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:02.153585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:02.153585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:16:02.153585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:16:02.153585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:16:02.153585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:16:02.153585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:16:02.153585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:16:02.153585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:16:02.153585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:02.153585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 480/957] 10.222.0.24 () {56 vars in 1120 bytes} [Fri May 27 03:16:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:16:02.158828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc5cdc-584b-433f-a136-20b52d420878 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:02.181757 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:02.271689 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc5cdc-584b-433f-a136-20b52d420878 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:02.311068 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74f8267-0141-442d-a7bf-39849c38aa7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:02.311738 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f8267-0141-442d-a7bf-39849c38aa7b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:02.312018 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f8267-0141-442d-a7bf-39849c38aa7b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:02.312294 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74f8267-0141-442d-a7bf-39849c38aa7b None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:02.313103 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f8267-0141-442d-a7bf-39849c38aa7b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:02.313412 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f8267-0141-442d-a7bf-39849c38aa7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:02.314788 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74f8267-0141-442d-a7bf-39849c38aa7b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:02.324159 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 478/958] 10.222.0.24 () {56 vars in 1115 bytes} [Fri May 27 03:16:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:02.329622 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7cd2f6-b613-4c30-ad52-570937c88ab6 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:02.375739 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:02.389048 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbcd53-9f19-4847-aa98-69b03d2296ab None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:02.389048 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbcd53-9f19-4847-aa98-69b03d2296ab None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:02.389048 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badbcd53-9f19-4847-aa98-69b03d2296ab None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:02.389048 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 479/959] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:16:02.390263 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fff4ca-0f9c-42e1-a7ff-9689bda4057d None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:02.390742 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fff4ca-0f9c-42e1-a7ff-9689bda4057d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:02.391271 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fff4ca-0f9c-42e1-a7ff-9689bda4057d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:02.408098 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-16fff4ca-0f9c-42e1-a7ff-9689bda4057d None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:02.409612 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-16fff4ca-0f9c-42e1-a7ff-9689bda4057d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:02.441561 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7cd2f6-b613-4c30-ad52-570937c88ab6 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:02.477281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae3f854-a1c2-4676-b686-00ccfd6eef01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:02.478127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3f854-a1c2-4676-b686-00ccfd6eef01 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:02.478672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3f854-a1c2-4676-b686-00ccfd6eef01 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:02.478672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3f854-a1c2-4676-b686-00ccfd6eef01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:02.489615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae3f854-a1c2-4676-b686-00ccfd6eef01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:02.491091 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae3f854-a1c2-4676-b686-00ccfd6eef01 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:02.495034 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 480/960] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:16:02.495034 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a9cbc-27cd-4a5a-b9fd-522792bea9eb None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:02.495034 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a9cbc-27cd-4a5a-b9fd-522792bea9eb None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:02.495034 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a9cbc-27cd-4a5a-b9fd-522792bea9eb None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:02.499944 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 481/961] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:02.501305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c53ab3-6b21-4339-9e18-37230ca5d905 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:02.501430 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c53ab3-6b21-4339-9e18-37230ca5d905 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:02.501635 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c53ab3-6b21-4339-9e18-37230ca5d905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:02.512786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-6ae3f854-a1c2-4676-b686-00ccfd6eef01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:16:02.512786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:02.512786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:02.512786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:02.512786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:02.512786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:02.512786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:02.512786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:02.512786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:02.512786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:02.512786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:02.512786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:02.512786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:16:02.512786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:16:02.512786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:16:02.513855 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:16:02.513855 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:02.513855 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:02.513855 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:02.513855 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:02.513855 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:02.513855 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:02.513855 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:02.513855 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:16:02.513855 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:02.513855 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:02.513855 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:02.513855 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:02.513855 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:02.513855 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:02.513855 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:02.513855 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:02.514988 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:16:02.514988 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:16:02.514988 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:16:02.514988 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:02.515257 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:02.520996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 481/962] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:16:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:16:02.520996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae3f854-a1c2-4676-b686-00ccfd6eef01 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:02.522401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-12c53ab3-6b21-4339-9e18-37230ca5d905 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:02.523640 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-12c53ab3-6b21-4339-9e18-37230ca5d905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:02.593245 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 482/963] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:16:02.594347 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c53ab3-6b21-4339-9e18-37230ca5d905 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:02.623150 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:02.631028 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae3f854-a1c2-4676-b686-00ccfd6eef01 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:02.666492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614847d-999f-4eae-81c0-aa457e42b0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:02.667255 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614847d-999f-4eae-81c0-aa457e42b0c3 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:02.667541 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614847d-999f-4eae-81c0-aa457e42b0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:02.667820 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614847d-999f-4eae-81c0-aa457e42b0c3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:02.668477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4614847d-999f-4eae-81c0-aa457e42b0c3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:02.669847 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4614847d-999f-4eae-81c0-aa457e42b0c3 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:02.682510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-4614847d-999f-4eae-81c0-aa457e42b0c3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:16:02.682510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:02.682510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:16:02.682510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:16:02.682510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:16:02.682510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:16:02.682510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:16:02.682510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:16:02.682510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:16:02.682510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:02.682510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:16:02.682510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:02.682510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:02.682510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:02.682510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:02.682510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:02.682510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:02.683575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:02.683575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:02.683575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:02.683575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:02.683575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:02.683575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:02.683575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:16:02.683575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:16:02.683575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:16:02.683575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:16:02.683575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:02.683575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:02.683575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:02.683575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:02.683575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:02.683575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:02.684452 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:02.684452 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:16:02.684452 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:02.684452 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:02.684452 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:02.684452 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:02.684452 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:02.684452 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:02.684452 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:02.684452 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:02.684452 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:16:02.684452 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:16:02.684452 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:16:02.684452 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:02.684452 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 482/964] 10.222.0.24 () {56 vars in 1126 bytes} [Fri May 27 03:16:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:16:02.685247 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614847d-999f-4eae-81c0-aa457e42b0c3 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:02.717851 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c53ab3-6b21-4339-9e18-37230ca5d905 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:02.755867 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998508a-0304-4596-ab03-55371727937c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:02.756852 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998508a-0304-4596-ab03-55371727937c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:02.757407 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998508a-0304-4596-ab03-55371727937c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:02.757796 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6998508a-0304-4596-ab03-55371727937c None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:02.758583 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6998508a-0304-4596-ab03-55371727937c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:02.758987 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6998508a-0304-4596-ab03-55371727937c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:02.760411 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6998508a-0304-4596-ab03-55371727937c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:02.761946 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:02.772429 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 483/965] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:02.781679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998508a-0304-4596-ab03-55371727937c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:02.802324 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614847d-999f-4eae-81c0-aa457e42b0c3 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:02.839790 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111810b-8496-479e-aeac-341487cc8186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:02.840822 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111810b-8496-479e-aeac-341487cc8186 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:02.841314 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111810b-8496-479e-aeac-341487cc8186 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:02.841777 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e111810b-8496-479e-aeac-341487cc8186 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:02.842641 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111810b-8496-479e-aeac-341487cc8186 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:02.844260 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e111810b-8496-479e-aeac-341487cc8186 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:02.845756 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: WARNING keystone.server.flask.common [None req-e111810b-8496-479e-aeac-341487cc8186 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 27 03:16:02.876538 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6998508a-0304-4596-ab03-55371727937c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:02.892972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.sql.core [None req-e111810b-8496-479e-aeac-341487cc8186 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 27 03:16:02.892972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 27 03:16:02.892972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [parameters: {'user_id': '720af87df48b4b2f9f973558ca4a301f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 27 03:16:02.892972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80481) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 27 03:16:02.909399 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-e111810b-8496-479e-aeac-341487cc8186 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 27 03:16:02.909399 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:02.909399 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 03:16:02.909399 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 03:16:02.909399 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 03:16:02.909399 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 03:16:02.909399 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 03:16:02.909399 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application result = self._query(query) May 27 03:16:02.909399 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 03:16:02.909399 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application conn.query(q) May 27 03:16:02.909399 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 03:16:02.909399 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 03:16:02.909399 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 03:16:02.909399 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application result.read() May 27 03:16:02.909399 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 03:16:02.910297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 03:16:02.910297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 03:16:02.910297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 03:16:02.910297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 03:16:02.910297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 03:16:02.910297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 03:16:02.910297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 03:16:02.910297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 27 03:16:02.910297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:02.910297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 27 03:16:02.910297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:02.910297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:02.910297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 27 03:16:02.910297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 27 03:16:02.910297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 27 03:16:02.910297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 27 03:16:02.910297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 03:16:02.910297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application next(self.gen) May 27 03:16:02.911378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 27 03:16:02.911378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application yield resource May 27 03:16:02.911378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 03:16:02.911378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application next(self.gen) May 27 03:16:02.911378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 27 03:16:02.911378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self.session.rollback() May 27 03:16:02.911378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 27 03:16:02.911378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self.force_reraise() May 27 03:16:02.911378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 27 03:16:02.911378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise self.value May 27 03:16:02.911378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 27 03:16:02.911378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 27 03:16:02.911378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 27 03:16:02.911378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application session.commit() May 27 03:16:02.911378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 27 03:16:02.911378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 27 03:16:02.911378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 27 03:16:02.912265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._prepare_impl() May 27 03:16:02.912265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 27 03:16:02.912265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self.session.flush() May 27 03:16:02.912265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 27 03:16:02.912265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._flush(objects) May 27 03:16:02.912265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 27 03:16:02.912265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 27 03:16:02.912265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 27 03:16:02.912265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application compat.raise_( May 27 03:16:02.912265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 03:16:02.912265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception May 27 03:16:02.912265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 27 03:16:02.912265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application flush_context.execute() May 27 03:16:02.912265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 27 03:16:02.912265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rec.execute(self) May 27 03:16:02.912265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 27 03:16:02.912265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 27 03:16:02.913163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 27 03:16:02.913163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application _emit_insert_statements( May 27 03:16:02.913163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 27 03:16:02.913163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application result = connection._execute_20( May 27 03:16:02.913163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 27 03:16:02.913163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 27 03:16:02.913163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 27 03:16:02.913163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 27 03:16:02.913163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 27 03:16:02.913163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ret = self._execute_context( May 27 03:16:02.913163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 27 03:16:02.913163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 27 03:16:02.913163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 27 03:16:02.913163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 27 03:16:02.913163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 03:16:02.913163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception May 27 03:16:02.914603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 03:16:02.914603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 03:16:02.914603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 03:16:02.914603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 03:16:02.914603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 03:16:02.914603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application result = self._query(query) May 27 03:16:02.914603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 03:16:02.914603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application conn.query(q) May 27 03:16:02.914603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 03:16:02.914603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 03:16:02.914603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 03:16:02.914603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application result.read() May 27 03:16:02.914603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 03:16:02.914603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 03:16:02.914603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 03:16:02.914603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 03:16:02.917407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 03:16:02.917407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 03:16:02.917407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 03:16:02.917407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 03:16:02.917407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 27 03:16:02.917407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 27 03:16:02.917407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [parameters: {'user_id': '720af87df48b4b2f9f973558ca4a301f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 27 03:16:02.917407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 27 03:16:02.917407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:02.917407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:16:02.917407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:02.917407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:02.917407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:02.917407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:02.917407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:02.918904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:02.918904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:02.918904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:02.918904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:02.918904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:02.918904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:02.918904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:02.918904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 27 03:16:02.918904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 27 03:16:02.918904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:02.918904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:02.918904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:16:02.918904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:16:02.918904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:16:02.918904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:16:02.918904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 27 03:16:02.918904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 27 03:16:02.920056 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 27 03:16:02.920056 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 27 03:16:02.920056 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 27 03:16:02.920056 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 27 03:16:02.920056 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 27 03:16:02.920056 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:02.920056 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 483/966] 10.222.0.24 () {60 vars in 1131 bytes} [Fri May 27 03:16:02 2022] POST /identity/v3/users => generated 160 bytes in 226 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 27 03:16:02.920056 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111810b-8496-479e-aeac-341487cc8186 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:02.921530 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968681f0-1449-4b4f-9720-4b60c2a7a3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:02.922210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968681f0-1449-4b4f-9720-4b60c2a7a3d8 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:02.922488 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968681f0-1449-4b4f-9720-4b60c2a7a3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:02.922763 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968681f0-1449-4b4f-9720-4b60c2a7a3d8 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:02.923501 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968681f0-1449-4b4f-9720-4b60c2a7a3d8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:02.923805 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968681f0-1449-4b4f-9720-4b60c2a7a3d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:02.925200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968681f0-1449-4b4f-9720-4b60c2a7a3d8 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:02.930618 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 484/967] 10.222.0.24 () {56 vars in 1122 bytes} [Fri May 27 03:16:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:02.935808 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968681f0-1449-4b4f-9720-4b60c2a7a3d8 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:02.941950 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:02.998062 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:03.024456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e111810b-8496-479e-aeac-341487cc8186 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:03.049859 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968681f0-1449-4b4f-9720-4b60c2a7a3d8 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:03.066943 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f697ec-db07-4e4b-9222-88747b03f157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:03.068122 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f697ec-db07-4e4b-9222-88747b03f157 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:03.068823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f697ec-db07-4e4b-9222-88747b03f157 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:03.069256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f697ec-db07-4e4b-9222-88747b03f157 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:03.083267 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f697ec-db07-4e4b-9222-88747b03f157 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:03.085006 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f697ec-db07-4e4b-9222-88747b03f157 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:03.088097 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c4219-e525-417e-94fc-187fc44798fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:03.088965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c4219-e525-417e-94fc-187fc44798fb None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:03.089317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c4219-e525-417e-94fc-187fc44798fb None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:03.089657 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c4219-e525-417e-94fc-187fc44798fb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:03.097591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c4219-e525-417e-94fc-187fc44798fb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:03.099159 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c4219-e525-417e-94fc-187fc44798fb None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:03.101984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-e6f697ec-db07-4e4b-9222-88747b03f157 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:16:03.101984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:03.101984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:03.101984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:03.101984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:03.101984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:03.101984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:03.101984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:03.101984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:03.101984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:03.101984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:03.101984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:03.101984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:16:03.101984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:16:03.101984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:16:03.101984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:16:03.102954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:03.102954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:03.102954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:16:03.102954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:16:03.102954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:16:03.102954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:16:03.102954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:03.102954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:03.102954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:03.102954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:03.102954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:03.102954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:16:03.102954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:03.102954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:03.102954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:03.102954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:03.102954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:03.102954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:03.104392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:03.104392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:03.104392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:16:03.104392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:16:03.104392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:16:03.104392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:16:03.104392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:16:03.104392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:16:03.104392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:16:03.104392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:03.104392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 484/968] 10.222.0.24 () {56 vars in 1117 bytes} [Fri May 27 03:16:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:16:03.104988 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f697ec-db07-4e4b-9222-88747b03f157 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:03.117101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-7e8c4219-e525-417e-94fc-187fc44798fb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:16:03.117101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:03.117101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:03.117101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:03.117101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:03.117101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:03.117101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:03.117101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:03.117101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:03.117101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:03.117101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:03.117101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:03.117101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:16:03.117101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:16:03.117101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:16:03.117101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:16:03.118272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:03.118272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:03.118272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:16:03.118272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:16:03.118272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:16:03.118272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:16:03.118272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:03.118272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:03.118272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:03.118272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:03.118272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:03.118272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:16:03.118272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:03.118272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:03.118272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:03.118272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:03.118272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:03.118272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:03.119482 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:03.119482 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:03.119482 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:16:03.119482 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:16:03.119482 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:16:03.119482 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:16:03.119482 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:16:03.119482 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:16:03.119482 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:16:03.119482 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:16:03.120685 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 485/969] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:16:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:16:03.121336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c4219-e525-417e-94fc-187fc44798fb None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:03.200616 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:03.224624 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f697ec-db07-4e4b-9222-88747b03f157 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:03.233866 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c4219-e525-417e-94fc-187fc44798fb None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:03.250365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:03.261993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7bb71-45d8-49e7-9f1f-f268700f5c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:03.262648 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7bb71-45d8-49e7-9f1f-f268700f5c6a None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:03.262953 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7bb71-45d8-49e7-9f1f-f268700f5c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:03.263228 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7bb71-45d8-49e7-9f1f-f268700f5c6a None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:03.263928 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf7bb71-45d8-49e7-9f1f-f268700f5c6a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:03.264230 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf7bb71-45d8-49e7-9f1f-f268700f5c6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:03.265593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf7bb71-45d8-49e7-9f1f-f268700f5c6a None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:03.268468 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e941143-2162-4e43-9b73-8a7f6bf9e5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:03.269131 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e941143-2162-4e43-9b73-8a7f6bf9e5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:03.269411 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e941143-2162-4e43-9b73-8a7f6bf9e5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:03.269682 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e941143-2162-4e43-9b73-8a7f6bf9e5e6 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:03.270509 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e941143-2162-4e43-9b73-8a7f6bf9e5e6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:03.270895 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e941143-2162-4e43-9b73-8a7f6bf9e5e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:03.272312 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e941143-2162-4e43-9b73-8a7f6bf9e5e6 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:03.278946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 485/970] 10.222.0.24 () {56 vars in 1122 bytes} [Fri May 27 03:16:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:03.278946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7bb71-45d8-49e7-9f1f-f268700f5c6a None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:03.299982 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 486/971] 10.222.0.24 () {56 vars in 1149 bytes} [Fri May 27 03:16:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:03.301334 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e941143-2162-4e43-9b73-8a7f6bf9e5e6 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:03.386194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7bb71-45d8-49e7-9f1f-f268700f5c6a None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:03.402487 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e941143-2162-4e43-9b73-8a7f6bf9e5e6 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:03.419178 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11992b-df28-45a7-929f-0a887f8fa8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:03.419853 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11992b-df28-45a7-929f-0a887f8fa8d7 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:03.420134 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11992b-df28-45a7-929f-0a887f8fa8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:03.420410 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11992b-df28-45a7-929f-0a887f8fa8d7 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:03.421215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11992b-df28-45a7-929f-0a887f8fa8d7 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:03.421520 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11992b-df28-45a7-929f-0a887f8fa8d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:03.422968 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11992b-df28-45a7-929f-0a887f8fa8d7 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:03.436448 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5ebb4-b01e-4a86-a2dd-be0447f5f076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:03.437095 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5ebb4-b01e-4a86-a2dd-be0447f5f076 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:03.437374 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5ebb4-b01e-4a86-a2dd-be0447f5f076 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:03.437644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5ebb4-b01e-4a86-a2dd-be0447f5f076 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:03.438364 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f5ebb4-b01e-4a86-a2dd-be0447f5f076 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:03.439751 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f5ebb4-b01e-4a86-a2dd-be0447f5f076 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:03.461172 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 486/972] 10.222.0.24 () {56 vars in 1101 bytes} [Fri May 27 03:16:03 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:03.461172 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11992b-df28-45a7-929f-0a887f8fa8d7 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:03.461822 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:03.542892 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 487/973] 10.222.0.24 () {60 vars in 1144 bytes} [Fri May 27 03:16:03 2022] POST /identity/v3/endpoints => generated 349 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:16:03.542892 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747f781-5011-4fcc-a758-de7d4f97a139 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:03.542892 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747f781-5011-4fcc-a758-de7d4f97a139 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:03.542892 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747f781-5011-4fcc-a758-de7d4f97a139 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:03.542892 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 488/974] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:03.550049 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e3c0a-a895-4263-8bbb-7b9dbe698d99 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:03.550336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e3c0a-a895-4263-8bbb-7b9dbe698d99 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:03.550618 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e3c0a-a895-4263-8bbb-7b9dbe698d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:03.571370 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-b37e3c0a-a895-4263-8bbb-7b9dbe698d99 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:03.572635 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b37e3c0a-a895-4263-8bbb-7b9dbe698d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:03.578775 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11992b-df28-45a7-929f-0a887f8fa8d7 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:03.614196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:03.631582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a86c0-cde6-4e4e-bed4-ef4c6f1947c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:03.633344 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a86c0-cde6-4e4e-bed4-ef4c6f1947c7 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:03.633887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a86c0-cde6-4e4e-bed4-ef4c6f1947c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:03.634369 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5a86c0-cde6-4e4e-bed4-ef4c6f1947c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:03.671422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 489/975] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:16:03.692932 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e3c0a-a895-4263-8bbb-7b9dbe698d99 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:03.716067 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:03.726084 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a86c0-cde6-4e4e-bed4-ef4c6f1947c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:03.734306 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5a86c0-cde6-4e4e-bed4-ef4c6f1947c7 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:03.817059 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e3c0a-a895-4263-8bbb-7b9dbe698d99 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:03.848950 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 487/976] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:16:03 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:16:03.854682 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a86c0-cde6-4e4e-bed4-ef4c6f1947c7 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:03.856814 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152747a5-04e0-48ec-a1ad-5e2bab67591a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:03.857606 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152747a5-04e0-48ec-a1ad-5e2bab67591a None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:03.857985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152747a5-04e0-48ec-a1ad-5e2bab67591a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:03.858353 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152747a5-04e0-48ec-a1ad-5e2bab67591a None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:03.859227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152747a5-04e0-48ec-a1ad-5e2bab67591a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:03.860692 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152747a5-04e0-48ec-a1ad-5e2bab67591a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:03.942434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 490/977] 10.222.0.24 () {60 vars in 1141 bytes} [Fri May 27 03:16:03 2022] POST /identity/v3/services => generated 242 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:16:03.950252 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:03.954470 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5a86c0-cde6-4e4e-bed4-ef4c6f1947c7 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:04.003039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72803574-7278-44c7-82ad-773bcc9a3112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:04.003770 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72803574-7278-44c7-82ad-773bcc9a3112 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:04.004046 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72803574-7278-44c7-82ad-773bcc9a3112 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:04.004314 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72803574-7278-44c7-82ad-773bcc9a3112 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:04.016117 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72803574-7278-44c7-82ad-773bcc9a3112 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:04.017587 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72803574-7278-44c7-82ad-773bcc9a3112 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:04.039016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-72803574-7278-44c7-82ad-773bcc9a3112 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:16:04.039016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:04.039016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:04.039016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:04.039016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:04.039016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:04.039016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:04.039016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:04.039016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:04.039016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:04.039016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:04.039016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:04.039016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:16:04.039016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:16:04.039016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:16:04.041105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:16:04.041105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:04.041105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:04.041105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:04.041105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:04.041105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:04.041105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:04.041105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:04.041105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:16:04.041105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:04.041105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:04.041105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:04.041105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:04.041105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:04.041105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:04.041105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:04.041105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:04.042277 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:16:04.042277 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:16:04.042277 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:16:04.042277 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:04.049697 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 488/978] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:16:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:16:04.051289 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152747a5-04e0-48ec-a1ad-5e2bab67591a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:04.087669 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:04.170515 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152747a5-04e0-48ec-a1ad-5e2bab67591a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:04.206097 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003fe994-c4a8-40be-a750-a9fa2df086a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:04.206822 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fe994-c4a8-40be-a750-a9fa2df086a0 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:04.207141 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fe994-c4a8-40be-a750-a9fa2df086a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:04.207416 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fe994-c4a8-40be-a750-a9fa2df086a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:04.208139 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003fe994-c4a8-40be-a750-a9fa2df086a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:04.208448 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003fe994-c4a8-40be-a750-a9fa2df086a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:04.209815 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003fe994-c4a8-40be-a750-a9fa2df086a0 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:04.221774 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 491/979] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:16:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:16:04.227657 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72803574-7278-44c7-82ad-773bcc9a3112 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:04.329041 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72803574-7278-44c7-82ad-773bcc9a3112 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:04.363612 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e547863-9576-4b8f-98ba-88e4f8c14cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:04.364405 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e547863-9576-4b8f-98ba-88e4f8c14cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:04.364688 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e547863-9576-4b8f-98ba-88e4f8c14cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:04.364973 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e547863-9576-4b8f-98ba-88e4f8c14cc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:04.371741 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e547863-9576-4b8f-98ba-88e4f8c14cc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:04.373181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e547863-9576-4b8f-98ba-88e4f8c14cc4 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:04.386864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-8e547863-9576-4b8f-98ba-88e4f8c14cc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:16:04.386864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:04.386864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:04.386864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:04.386864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:04.386864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:04.386864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:04.386864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:04.386864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:04.386864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:04.386864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:04.386864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:04.386864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:16:04.386864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:16:04.386864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:16:04.388053 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:16:04.388053 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:04.388053 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:04.388053 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:04.388053 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:04.388053 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:04.388053 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:04.388053 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:04.388053 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:16:04.388053 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:04.388053 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:04.388053 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:04.388053 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:04.388053 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:04.388053 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:04.388053 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:04.388053 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:04.389126 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:16:04.389126 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:16:04.389126 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:16:04.389126 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:16:04.389126 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:16:04.389126 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:16:04.389126 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:16:04.389126 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:04.390922 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 489/980] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:16:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:16:04.394016 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003fe994-c4a8-40be-a750-a9fa2df086a0 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:04.480495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:04.498021 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003fe994-c4a8-40be-a750-a9fa2df086a0 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:04.532371 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa6974-c14d-44b3-a67d-6866b4a6f4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:04.533026 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa6974-c14d-44b3-a67d-6866b4a6f4b6 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:04.533311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa6974-c14d-44b3-a67d-6866b4a6f4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:04.533585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa6974-c14d-44b3-a67d-6866b4a6f4b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:04.534376 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa6974-c14d-44b3-a67d-6866b4a6f4b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:04.534732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa6974-c14d-44b3-a67d-6866b4a6f4b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:04.536195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa6974-c14d-44b3-a67d-6866b4a6f4b6 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:04.548337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 492/981] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:16:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:04.554073 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e547863-9576-4b8f-98ba-88e4f8c14cc4 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:04.564106 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:04.655552 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e547863-9576-4b8f-98ba-88e4f8c14cc4 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:04.688211 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9014c-0ee9-461b-9efd-36a8f5dd91e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:04.688877 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9014c-0ee9-461b-9efd-36a8f5dd91e9 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:04.689159 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9014c-0ee9-461b-9efd-36a8f5dd91e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:04.689434 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9014c-0ee9-461b-9efd-36a8f5dd91e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:04.690273 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd9014c-0ee9-461b-9efd-36a8f5dd91e9 None admin] RBAC: Adding query filter params (role.id=1705f116742c4217bb8f6eab32b40850, scope.project.id=6ceffdd955324aec84ba048d2137ae00, user.id=07c9688ca035484ba5341859574c2799) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:04.690578 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd9014c-0ee9-461b-9efd-36a8f5dd91e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:04.692070 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd9014c-0ee9-461b-9efd-36a8f5dd91e9 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:04.706868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 490/982] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:16:04 2022] GET /identity/v3/role_assignments?user.id=07c9688ca035484ba5341859574c2799&scope.project.id=6ceffdd955324aec84ba048d2137ae00&role.id=1705f116742c4217bb8f6eab32b40850 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:05.842074 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa945e8d-034d-45f7-8684-6509d93a42b9 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:05.842612 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa945e8d-034d-45f7-8684-6509d93a42b9 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:05.842612 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa945e8d-034d-45f7-8684-6509d93a42b9 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:05.844253 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 493/983] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:05.850045 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907369b-fc1e-4d82-8892-3c5d1ca6cc17 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:05.850609 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907369b-fc1e-4d82-8892-3c5d1ca6cc17 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:05.851354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8907369b-fc1e-4d82-8892-3c5d1ca6cc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:05.868143 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-8907369b-fc1e-4d82-8892-3c5d1ca6cc17 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:05.869652 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8907369b-fc1e-4d82-8892-3c5d1ca6cc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:05.936051 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:05.941031 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 491/984] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:05 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:16:05.964715 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa945e8d-034d-45f7-8684-6509d93a42b9 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:06.072468 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa945e8d-034d-45f7-8684-6509d93a42b9 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:06.113597 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50c27d-61e9-44cb-91eb-e30df5d2955d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:06.114333 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50c27d-61e9-44cb-91eb-e30df5d2955d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:06.114615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50c27d-61e9-44cb-91eb-e30df5d2955d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:06.114927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50c27d-61e9-44cb-91eb-e30df5d2955d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:06.115589 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af50c27d-61e9-44cb-91eb-e30df5d2955d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:06.117041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af50c27d-61e9-44cb-91eb-e30df5d2955d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:06.129804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-af50c27d-61e9-44cb-91eb-e30df5d2955d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:16:06.129804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:06.129804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:16:06.129804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:16:06.129804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:16:06.129804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:16:06.129804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:16:06.129804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:16:06.129804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:16:06.129804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:16:06.129804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:16:06.129804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:16:06.129804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:06.129804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:06.129804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:06.129804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:06.129804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:06.130893 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:06.130893 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:06.130893 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:06.130893 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:06.130893 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:06.130893 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:06.130893 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:16:06.130893 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:16:06.130893 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:16:06.130893 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:16:06.130893 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:06.130893 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:06.130893 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:06.130893 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:06.130893 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:06.130893 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:06.132317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:06.132317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:16:06.132317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:06.132317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:06.132317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:06.132317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:06.132317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:06.132317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:06.132317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:06.132317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:06.132317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:16:06.132317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:16:06.132317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:16:06.132317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:16:06.132317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 494/985] 10.222.0.24 () {56 vars in 1126 bytes} [Fri May 27 03:16:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:16:06.139659 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8907369b-fc1e-4d82-8892-3c5d1ca6cc17 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:06.169735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86268a9f-d1f4-4097-904e-ab3cb9d005c2 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:06.170022 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86268a9f-d1f4-4097-904e-ab3cb9d005c2 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:06.170300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86268a9f-d1f4-4097-904e-ab3cb9d005c2 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:06.171837 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 495/986] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:16:06.176492 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbaefb4-51fb-4fe4-8598-2324115dbf69 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:06.176777 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbaefb4-51fb-4fe4-8598-2324115dbf69 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:06.177057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbaefb4-51fb-4fe4-8598-2324115dbf69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:06.188334 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:06.193910 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-ecbaefb4-51fb-4fe4-8598-2324115dbf69 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:06.195669 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ecbaefb4-51fb-4fe4-8598-2324115dbf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:06.243715 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8907369b-fc1e-4d82-8892-3c5d1ca6cc17 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:06.269433 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 496/987] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:06 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:16:06.281807 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db3f0f-c510-47d9-80dc-0f17b8a68742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:06.282474 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db3f0f-c510-47d9-80dc-0f17b8a68742 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:06.282761 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db3f0f-c510-47d9-80dc-0f17b8a68742 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:06.283068 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db3f0f-c510-47d9-80dc-0f17b8a68742 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:06.283776 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db3f0f-c510-47d9-80dc-0f17b8a68742 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:06.284079 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db3f0f-c510-47d9-80dc-0f17b8a68742 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:06.285454 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db3f0f-c510-47d9-80dc-0f17b8a68742 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:06.290664 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 492/988] 10.222.0.24 () {56 vars in 1122 bytes} [Fri May 27 03:16:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:06.291838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbaefb4-51fb-4fe4-8598-2324115dbf69 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:06.295712 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db3f0f-c510-47d9-80dc-0f17b8a68742 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:06.360684 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:06.399736 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbaefb4-51fb-4fe4-8598-2324115dbf69 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:06.402959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db3f0f-c510-47d9-80dc-0f17b8a68742 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:06.434815 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b15965-d559-4da4-83c3-8e624545a8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:06.436100 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b15965-d559-4da4-83c3-8e624545a8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:06.436519 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b15965-d559-4da4-83c3-8e624545a8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:06.436947 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b15965-d559-4da4-83c3-8e624545a8c6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:06.437742 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fcc823-77f1-41e4-8fb4-35c3143b7753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:06.437988 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b15965-d559-4da4-83c3-8e624545a8c6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:06.438403 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fcc823-77f1-41e4-8fb4-35c3143b7753 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:06.438687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fcc823-77f1-41e4-8fb4-35c3143b7753 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:06.438996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fcc823-77f1-41e4-8fb4-35c3143b7753 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:06.439703 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fcc823-77f1-41e4-8fb4-35c3143b7753 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:06.440011 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fcc823-77f1-41e4-8fb4-35c3143b7753 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:06.440304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b15965-d559-4da4-83c3-8e624545a8c6 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:06.441363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fcc823-77f1-41e4-8fb4-35c3143b7753 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:06.443936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:06.447014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 493/989] 10.222.0.24 () {56 vars in 1122 bytes} [Fri May 27 03:16:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:16:06.452967 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fcc823-77f1-41e4-8fb4-35c3143b7753 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:06.462010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-d5b15965-d559-4da4-83c3-8e624545a8c6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:16:06.462010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:06.462010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:16:06.462010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:16:06.462010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:16:06.462010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:16:06.462010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:16:06.462010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:16:06.462010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:16:06.462010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:16:06.462010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:16:06.462010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:16:06.462010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:06.462010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:06.462010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:06.462010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:06.462010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:06.463222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:06.463222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:06.463222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:06.463222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:06.463222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:06.463222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:06.463222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:16:06.463222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:16:06.463222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:16:06.463222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:16:06.463222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:06.463222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:06.463222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:06.463222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:06.463222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:06.463222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:06.464278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:06.464278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:16:06.464278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:06.464278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:06.464278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:06.464278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:06.464278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:06.464278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:06.464278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:06.464278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:06.464278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:16:06.464278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:16:06.464278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:16:06.464278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:16:06.464278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 497/990] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:16:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:16:06.467725 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b15965-d559-4da4-83c3-8e624545a8c6 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:06.574768 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fcc823-77f1-41e4-8fb4-35c3143b7753 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:06.594502 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b15965-d559-4da4-83c3-8e624545a8c6 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:06.605153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:06.613920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165d247-489c-4e33-94bb-a0a1797ea9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:06.614754 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165d247-489c-4e33-94bb-a0a1797ea9bf None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:06.615243 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165d247-489c-4e33-94bb-a0a1797ea9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:06.615700 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165d247-489c-4e33-94bb-a0a1797ea9bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:06.617227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8165d247-489c-4e33-94bb-a0a1797ea9bf None admin] RBAC: Adding query filter params (interface=admin, service_id=541921ac6534459b9fbcf826bcd8e51d, region_id=RegionOne) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:06.617689 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8165d247-489c-4e33-94bb-a0a1797ea9bf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:06.619226 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8165d247-489c-4e33-94bb-a0a1797ea9bf None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:06.624413 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 494/991] 10.222.0.24 () {56 vars in 1249 bytes} [Fri May 27 03:16:06 2022] GET /identity/v3/endpoints?service_id=541921ac6534459b9fbcf826bcd8e51d&interface=admin®ion_id=RegionOne => generated 193 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:06.629692 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165d247-489c-4e33-94bb-a0a1797ea9bf None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:06.632803 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00592fe8-9215-4a27-b8da-4462005391a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:06.633642 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00592fe8-9215-4a27-b8da-4462005391a4 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:06.633921 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00592fe8-9215-4a27-b8da-4462005391a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:06.634194 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00592fe8-9215-4a27-b8da-4462005391a4 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:06.634998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00592fe8-9215-4a27-b8da-4462005391a4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:06.635309 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00592fe8-9215-4a27-b8da-4462005391a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:06.636668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00592fe8-9215-4a27-b8da-4462005391a4 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:06.645990 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 498/992] 10.222.0.24 () {56 vars in 1112 bytes} [Fri May 27 03:16:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:16:06.647485 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00592fe8-9215-4a27-b8da-4462005391a4 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:06.716372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:06.734821 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165d247-489c-4e33-94bb-a0a1797ea9bf None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:06.757582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00592fe8-9215-4a27-b8da-4462005391a4 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:06.783923 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3a0c1-62bc-493f-9809-ee79678d536f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:06.784608 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3a0c1-62bc-493f-9809-ee79678d536f None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:06.784926 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3a0c1-62bc-493f-9809-ee79678d536f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:06.785166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c3a0c1-62bc-493f-9809-ee79678d536f None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:06.785905 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c3a0c1-62bc-493f-9809-ee79678d536f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:06.787317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c3a0c1-62bc-493f-9809-ee79678d536f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:06.803019 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 495/993] 10.222.0.24 () {56 vars in 1087 bytes} [Fri May 27 03:16:06 2022] GET /identity/v3/services => generated 1727 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:16:06.807249 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcbfcd2-3909-4180-adf2-290afe92c50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:06.807946 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbfcd2-3909-4180-adf2-290afe92c50c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:06.808227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbfcd2-3909-4180-adf2-290afe92c50c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:06.808500 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbfcd2-3909-4180-adf2-290afe92c50c None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:06.809228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcbfcd2-3909-4180-adf2-290afe92c50c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:06.809536 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcbfcd2-3909-4180-adf2-290afe92c50c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:06.810985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcbfcd2-3909-4180-adf2-290afe92c50c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:06.816963 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 499/994] 10.222.0.24 () {56 vars in 1112 bytes} [Fri May 27 03:16:06 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:06.823434 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3a0c1-62bc-493f-9809-ee79678d536f None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:06.886900 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:06.935146 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c3a0c1-62bc-493f-9809-ee79678d536f None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:06.966969 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00501d-1b7f-4531-989b-47d72fda48d4 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:06.967478 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00501d-1b7f-4531-989b-47d72fda48d4 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:06.967478 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00501d-1b7f-4531-989b-47d72fda48d4 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:06.968987 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 500/995] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:16:06.974086 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e79f0-88f7-4065-8d25-b66554ea2612 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:06.974366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e79f0-88f7-4065-8d25-b66554ea2612 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:06.974638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e79f0-88f7-4065-8d25-b66554ea2612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:06.976079 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce59bb3-e6da-469d-ac29-4a98cd9cfe23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:06.976747 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce59bb3-e6da-469d-ac29-4a98cd9cfe23 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:06.977036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce59bb3-e6da-469d-ac29-4a98cd9cfe23 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:06.977310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce59bb3-e6da-469d-ac29-4a98cd9cfe23 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:06.978107 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce59bb3-e6da-469d-ac29-4a98cd9cfe23 None admin] RBAC: Adding query filter params (interface=public, service_id=a66fadbccdb54f948d94de4f89896318, region_id=RegionOne) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:06.978424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce59bb3-e6da-469d-ac29-4a98cd9cfe23 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:06.979841 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce59bb3-e6da-469d-ac29-4a98cd9cfe23 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:06.986016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 496/996] 10.222.0.24 () {56 vars in 1251 bytes} [Fri May 27 03:16:06 2022] GET /identity/v3/endpoints?service_id=a66fadbccdb54f948d94de4f89896318&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:06.992470 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce59bb3-e6da-469d-ac29-4a98cd9cfe23 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:06.993749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e44e79f0-88f7-4065-8d25-b66554ea2612 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:06.995108 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e44e79f0-88f7-4065-8d25-b66554ea2612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:07.066252 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 501/997] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:06 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:16:07.088773 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e79f0-88f7-4065-8d25-b66554ea2612 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:07.107627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce59bb3-e6da-469d-ac29-4a98cd9cfe23 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:07.128489 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:07.142403 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e23e8f-09f2-4318-a3cd-5aff4e31bf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:07.143193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e23e8f-09f2-4318-a3cd-5aff4e31bf23 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:07.143475 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e23e8f-09f2-4318-a3cd-5aff4e31bf23 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:07.143754 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e23e8f-09f2-4318-a3cd-5aff4e31bf23 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:07.144436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e23e8f-09f2-4318-a3cd-5aff4e31bf23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:07.146075 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e23e8f-09f2-4318-a3cd-5aff4e31bf23 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:07.151001 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:07.162880 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 497/998] 10.222.0.24 () {56 vars in 1087 bytes} [Fri May 27 03:16:06 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:16:07.211638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e79f0-88f7-4065-8d25-b66554ea2612 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:07.248662 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85205ee2-7120-4624-b77f-2c0ac82b50ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:07.249387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85205ee2-7120-4624-b77f-2c0ac82b50ce None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:07.249670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85205ee2-7120-4624-b77f-2c0ac82b50ce None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:07.249945 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85205ee2-7120-4624-b77f-2c0ac82b50ce None admin] PATH_INFO: `/v3/services/image` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:07.250606 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85205ee2-7120-4624-b77f-2c0ac82b50ce None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:07.252036 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85205ee2-7120-4624-b77f-2c0ac82b50ce None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:07.264577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-85205ee2-7120-4624-b77f-2c0ac82b50ce None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:16:07.264577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:07.264577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:16:07.264577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:16:07.264577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:16:07.264577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:16:07.264577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:16:07.264577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:16:07.264577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:16:07.264577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:16:07.264577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:16:07.264577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:16:07.264577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:07.264577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:07.264577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:07.264577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:07.264577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:07.265769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:07.265769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:07.265769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:07.265769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:07.265769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:07.265769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:07.265769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:16:07.265769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:16:07.265769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:16:07.265769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:16:07.265769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:07.265769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:07.265769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:07.265769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:07.265769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:07.265769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:07.266607 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:07.266607 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:16:07.266607 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:07.266607 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:07.266607 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:07.266607 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:07.266607 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:07.266607 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:07.266607 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:07.266607 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:07.266607 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:16:07.266607 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:16:07.266607 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:16:07.266607 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:16:07.270994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 502/999] 10.222.0.24 () {56 vars in 1105 bytes} [Fri May 27 03:16:07 2022] GET /identity/v3/services/image => generated 86 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:16:07.272817 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e23e8f-09f2-4318-a3cd-5aff4e31bf23 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:07.384432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e23e8f-09f2-4318-a3cd-5aff4e31bf23 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:07.418972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c3e87-93cf-4193-b8cc-762a0d8cfe6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:07.419947 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c3e87-93cf-4193-b8cc-762a0d8cfe6e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:07.420407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c3e87-93cf-4193-b8cc-762a0d8cfe6e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:07.420859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c3e87-93cf-4193-b8cc-762a0d8cfe6e None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:07.421734 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c3e87-93cf-4193-b8cc-762a0d8cfe6e None admin] RBAC: Adding query filter params (name=image) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:07.422241 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c3e87-93cf-4193-b8cc-762a0d8cfe6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:07.423817 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c3e87-93cf-4193-b8cc-762a0d8cfe6e None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:07.429025 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 498/1000] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:16:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:07.434286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85205ee2-7120-4624-b77f-2c0ac82b50ce None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:07.537194 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85205ee2-7120-4624-b77f-2c0ac82b50ce None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:07.560375 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:07.571484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f117e28-15b2-4a61-9c1a-73bfca2c3b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:07.577541 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f117e28-15b2-4a61-9c1a-73bfca2c3b74 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:07.577541 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f117e28-15b2-4a61-9c1a-73bfca2c3b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:07.577541 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f117e28-15b2-4a61-9c1a-73bfca2c3b74 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:07.577541 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f117e28-15b2-4a61-9c1a-73bfca2c3b74 None admin] RBAC: Adding query filter params (type=image) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:07.577541 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f117e28-15b2-4a61-9c1a-73bfca2c3b74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:07.577541 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f117e28-15b2-4a61-9c1a-73bfca2c3b74 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:07.580311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 503/1001] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:16:07 2022] GET /identity/v3/services?type=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:09.017839 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3ca65-3f6a-4e7e-aff8-720871a2eb6b None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:09.018413 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3ca65-3f6a-4e7e-aff8-720871a2eb6b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:09.018413 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3ca65-3f6a-4e7e-aff8-720871a2eb6b None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:09.019813 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 499/1002] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:09.024959 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44d8c0-9b30-4cca-90e2-a23b1202e0c5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:09.025369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44d8c0-9b30-4cca-90e2-a23b1202e0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:09.025565 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44d8c0-9b30-4cca-90e2-a23b1202e0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:09.042117 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-1e44d8c0-9b30-4cca-90e2-a23b1202e0c5 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:09.043316 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1e44d8c0-9b30-4cca-90e2-a23b1202e0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:09.118943 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 504/1003] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:09 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:16:09.141633 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3ca65-3f6a-4e7e-aff8-720871a2eb6b None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:09.194360 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:09.250748 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3ca65-3f6a-4e7e-aff8-720871a2eb6b None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:09.284593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b5d07-d964-4c81-9138-c283bc2b1b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:09.285411 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b5d07-d964-4c81-9138-c283bc2b1b87 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:09.285691 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b5d07-d964-4c81-9138-c283bc2b1b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:09.285965 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b5d07-d964-4c81-9138-c283bc2b1b87 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:09.286621 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b5d07-d964-4c81-9138-c283bc2b1b87 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:09.288526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b5d07-d964-4c81-9138-c283bc2b1b87 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:09.300658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-373b5d07-d964-4c81-9138-c283bc2b1b87 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:16:09.300658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:09.300658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:16:09.300658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:16:09.300658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:16:09.300658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:16:09.300658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:16:09.300658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:16:09.300658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:16:09.300658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:09.300658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:16:09.300658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:09.300658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:09.300658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:09.300658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:09.300658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:09.300658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:09.301649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:09.301649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:09.301649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:09.301649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:09.301649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:09.301649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:09.301649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:16:09.301649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:16:09.301649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:16:09.301649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:16:09.301649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:09.301649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:09.301649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:09.301649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:09.301649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:09.301649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:09.307696 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:09.307696 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:16:09.307696 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:09.307696 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:09.307696 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:09.307696 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:09.307696 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:09.307696 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:09.307696 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:09.307696 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:09.307696 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:16:09.307696 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:16:09.307696 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:16:09.307696 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:09.307696 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 500/1004] 10.222.0.24 () {56 vars in 1126 bytes} [Fri May 27 03:16:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:16:09.310233 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44d8c0-9b30-4cca-90e2-a23b1202e0c5 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:09.387444 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b452e19-8d64-4da8-8b23-23a33416ad69 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:09.387714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b452e19-8d64-4da8-8b23-23a33416ad69 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:09.387999 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b452e19-8d64-4da8-8b23-23a33416ad69 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:09.389501 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 501/1005] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:09.394289 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99d1b2-83b3-4679-8ec6-614f6a224f4d None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:09.394575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99d1b2-83b3-4679-8ec6-614f6a224f4d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:09.394866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99d1b2-83b3-4679-8ec6-614f6a224f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:09.408448 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44d8c0-9b30-4cca-90e2-a23b1202e0c5 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:09.411916 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-5c99d1b2-83b3-4679-8ec6-614f6a224f4d None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:09.413651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5c99d1b2-83b3-4679-8ec6-614f6a224f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:09.426162 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:09.445127 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68503beb-6f4b-46c7-9e66-69145c9ee84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:09.445856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68503beb-6f4b-46c7-9e66-69145c9ee84e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:09.446145 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68503beb-6f4b-46c7-9e66-69145c9ee84e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:09.446413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68503beb-6f4b-46c7-9e66-69145c9ee84e None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:09.447135 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68503beb-6f4b-46c7-9e66-69145c9ee84e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:09.447439 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68503beb-6f4b-46c7-9e66-69145c9ee84e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:09.448785 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68503beb-6f4b-46c7-9e66-69145c9ee84e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:09.454321 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 505/1006] 10.222.0.24 () {56 vars in 1122 bytes} [Fri May 27 03:16:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:09.460614 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68503beb-6f4b-46c7-9e66-69145c9ee84e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:09.492699 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 502/1007] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:09 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:16:09.514645 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99d1b2-83b3-4679-8ec6-614f6a224f4d None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:09.561084 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:09.577445 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68503beb-6f4b-46c7-9e66-69145c9ee84e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:09.615672 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c5ac4-36b9-4b87-bf85-d705559b1bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:09.616419 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c5ac4-36b9-4b87-bf85-d705559b1bfd None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:09.616645 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c5ac4-36b9-4b87-bf85-d705559b1bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:09.616917 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59c5ac4-36b9-4b87-bf85-d705559b1bfd None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:09.617618 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c5ac4-36b9-4b87-bf85-d705559b1bfd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:09.617920 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c5ac4-36b9-4b87-bf85-d705559b1bfd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:09.619311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59c5ac4-36b9-4b87-bf85-d705559b1bfd None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:09.624433 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 506/1008] 10.222.0.24 () {56 vars in 1122 bytes} [Fri May 27 03:16:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:16:09.629956 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c5ac4-36b9-4b87-bf85-d705559b1bfd None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:09.631771 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99d1b2-83b3-4679-8ec6-614f6a224f4d None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:09.672831 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a52e6d-cddf-46da-a143-58d83e9cbdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:09.673672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a52e6d-cddf-46da-a143-58d83e9cbdc8 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:09.673958 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a52e6d-cddf-46da-a143-58d83e9cbdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:09.674227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a52e6d-cddf-46da-a143-58d83e9cbdc8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:09.674936 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a52e6d-cddf-46da-a143-58d83e9cbdc8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:09.676300 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a52e6d-cddf-46da-a143-58d83e9cbdc8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:09.679062 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:09.687993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-01a52e6d-cddf-46da-a143-58d83e9cbdc8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:16:09.687993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:09.687993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:16:09.687993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:16:09.687993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:16:09.687993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:16:09.687993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:16:09.687993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:16:09.687993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:16:09.687993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:09.687993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:16:09.687993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:09.687993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:09.687993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:09.687993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:09.687993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:09.687993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:09.694796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:09.694796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:09.694796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:09.694796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:09.694796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:09.694796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:09.694796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:16:09.694796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:16:09.694796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:16:09.694796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:16:09.694796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:09.694796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:09.694796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:09.694796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:09.694796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:09.694796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:09.695733 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:09.695733 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:16:09.695733 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:09.695733 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:09.695733 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:09.695733 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:09.695733 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:09.695733 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:09.695733 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:09.695733 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:09.695733 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:16:09.695733 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:16:09.695733 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:16:09.695733 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:09.695733 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 503/1009] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:16:09 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:16:09.696538 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a52e6d-cddf-46da-a143-58d83e9cbdc8 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:09.738817 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59c5ac4-36b9-4b87-bf85-d705559b1bfd None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:09.781150 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd56a0b-c442-40d3-bae7-dfa3f7d741ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:09.781823 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd56a0b-c442-40d3-bae7-dfa3f7d741ba None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:09.782114 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd56a0b-c442-40d3-bae7-dfa3f7d741ba None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:09.782368 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd56a0b-c442-40d3-bae7-dfa3f7d741ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:09.783157 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd56a0b-c442-40d3-bae7-dfa3f7d741ba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:09.784653 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd56a0b-c442-40d3-bae7-dfa3f7d741ba None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:09.805621 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a52e6d-cddf-46da-a143-58d83e9cbdc8 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:09.812557 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:09.838639 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d6572-224b-46be-8f38-13b2e989145b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:09.839525 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d6572-224b-46be-8f38-13b2e989145b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:09.854347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d6572-224b-46be-8f38-13b2e989145b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:09.854347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d6572-224b-46be-8f38-13b2e989145b None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:09.854347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d6572-224b-46be-8f38-13b2e989145b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:09.854347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d6572-224b-46be-8f38-13b2e989145b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:09.854347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d6572-224b-46be-8f38-13b2e989145b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:09.854347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 504/1010] 10.222.0.24 () {56 vars in 1112 bytes} [Fri May 27 03:16:09 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:16:09.854897 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13686cb-78a2-4d4f-9801-bd04b371c3e7 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:09.854897 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13686cb-78a2-4d4f-9801-bd04b371c3e7 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:09.854897 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13686cb-78a2-4d4f-9801-bd04b371c3e7 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:09.854897 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 505/1011] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:09.854897 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13686cb-78a2-4d4f-9801-bd04b371c3e7 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:09.905919 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 507/1012] 10.222.0.24 () {60 vars in 1144 bytes} [Fri May 27 03:16:09 2022] POST /identity/v3/endpoints => generated 325 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:16:09.905919 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255ddf4-16d8-48eb-aeea-386a9085ec49 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:09.905919 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255ddf4-16d8-48eb-aeea-386a9085ec49 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:09.905919 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b255ddf4-16d8-48eb-aeea-386a9085ec49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:09.924526 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-b255ddf4-16d8-48eb-aeea-386a9085ec49 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:09.925755 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b255ddf4-16d8-48eb-aeea-386a9085ec49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:09.968418 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13686cb-78a2-4d4f-9801-bd04b371c3e7 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:10.005193 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 508/1013] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:09 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:16:10.008128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30052ed1-479c-4d99-aba1-c131d24e4f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:10.008803 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30052ed1-479c-4d99-aba1-c131d24e4f53 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:10.009141 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30052ed1-479c-4d99-aba1-c131d24e4f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:10.009410 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30052ed1-479c-4d99-aba1-c131d24e4f53 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:10.010127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30052ed1-479c-4d99-aba1-c131d24e4f53 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:10.010474 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30052ed1-479c-4d99-aba1-c131d24e4f53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:10.011840 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30052ed1-479c-4d99-aba1-c131d24e4f53 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:10.022825 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 506/1014] 10.222.0.24 () {56 vars in 1112 bytes} [Fri May 27 03:16:09 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:16:10.024873 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b255ddf4-16d8-48eb-aeea-386a9085ec49 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:10.030503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30052ed1-479c-4d99-aba1-c131d24e4f53 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:10.064574 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:10.070356 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:10.139132 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b255ddf4-16d8-48eb-aeea-386a9085ec49 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:10.166591 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30052ed1-479c-4d99-aba1-c131d24e4f53 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:10.176592 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0b85c-b442-42a2-b979-ee0af2fca26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:10.177255 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0b85c-b442-42a2-b979-ee0af2fca26c None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:10.177542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0b85c-b442-42a2-b979-ee0af2fca26c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:10.177818 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0b85c-b442-42a2-b979-ee0af2fca26c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:10.178598 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa0b85c-b442-42a2-b979-ee0af2fca26c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:10.180061 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa0b85c-b442-42a2-b979-ee0af2fca26c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:10.203764 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614af99-74ad-40cc-888a-132df50c4f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:10.204435 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614af99-74ad-40cc-888a-132df50c4f60 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:10.204718 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614af99-74ad-40cc-888a-132df50c4f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:10.204992 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614af99-74ad-40cc-888a-132df50c4f60 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:10.205712 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c614af99-74ad-40cc-888a-132df50c4f60 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:10.207117 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c614af99-74ad-40cc-888a-132df50c4f60 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:10.390785 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 509/1015] 10.222.0.24 () {60 vars in 1144 bytes} [Fri May 27 03:16:10 2022] POST /identity/v3/endpoints => generated 334 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:16:10.410871 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 507/1016] 10.222.0.24 () {60 vars in 1141 bytes} [Fri May 27 03:16:10 2022] POST /identity/v3/services => generated 242 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:16:12.151391 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da3a01-0d18-4f8b-84b5-5f5eb8596137 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:12.151935 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da3a01-0d18-4f8b-84b5-5f5eb8596137 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:12.151935 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da3a01-0d18-4f8b-84b5-5f5eb8596137 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:12.153274 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 510/1017] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:12.158914 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20f0f3-29d3-4151-87ae-3a0a4505723e None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:12.159355 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20f0f3-29d3-4151-87ae-3a0a4505723e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:12.159528 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20f0f3-29d3-4151-87ae-3a0a4505723e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:12.176512 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-da20f0f3-29d3-4151-87ae-3a0a4505723e None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:12.177702 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-da20f0f3-29d3-4151-87ae-3a0a4505723e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:12.253031 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 508/1018] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:12 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:16:12.275925 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da3a01-0d18-4f8b-84b5-5f5eb8596137 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:12.368334 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:12.378269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da3a01-0d18-4f8b-84b5-5f5eb8596137 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:12.411002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843332f-218a-4ace-aa17-3d2bffef06d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:12.411792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843332f-218a-4ace-aa17-3d2bffef06d7 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:12.412073 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843332f-218a-4ace-aa17-3d2bffef06d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:12.412345 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843332f-218a-4ace-aa17-3d2bffef06d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:12.414038 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843332f-218a-4ace-aa17-3d2bffef06d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:12.415508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843332f-218a-4ace-aa17-3d2bffef06d7 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:12.419169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 511/1019] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:16:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:12.422969 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20f0f3-29d3-4151-87ae-3a0a4505723e None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:12.441524 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:12.528307 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20f0f3-29d3-4151-87ae-3a0a4505723e None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:12.567730 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e512cf0-bdda-4d6e-a21f-e33269af5212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:12.568401 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e512cf0-bdda-4d6e-a21f-e33269af5212 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:12.568684 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e512cf0-bdda-4d6e-a21f-e33269af5212 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:12.568992 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e512cf0-bdda-4d6e-a21f-e33269af5212 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:12.569723 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e512cf0-bdda-4d6e-a21f-e33269af5212 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:12.572112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e512cf0-bdda-4d6e-a21f-e33269af5212 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:12.670951 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 509/1020] 10.222.0.24 () {60 vars in 1141 bytes} [Fri May 27 03:16:12 2022] POST /identity/v3/projects => generated 311 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:16:12.684379 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396647e5-b5cc-4d99-bd04-ccd5b6c0f184 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:12.686365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396647e5-b5cc-4d99-bd04-ccd5b6c0f184 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:12.686365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396647e5-b5cc-4d99-bd04-ccd5b6c0f184 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:12.686365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 512/1021] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:12.692639 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c120ef-bfdc-49fa-8201-2a6141033a87 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:12.692942 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c120ef-bfdc-49fa-8201-2a6141033a87 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:12.693205 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c120ef-bfdc-49fa-8201-2a6141033a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:12.711875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-65c120ef-bfdc-49fa-8201-2a6141033a87 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:12.713115 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-65c120ef-bfdc-49fa-8201-2a6141033a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:12.726671 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5c1a0-7974-4f35-968f-3c5ec193a026 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:12.727060 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5c1a0-7974-4f35-968f-3c5ec193a026 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:12.727245 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5c1a0-7974-4f35-968f-3c5ec193a026 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:12.728652 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 513/1022] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:16:12.734364 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f08cb1-871e-4f39-bf35-bfe2913a6dc9 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:12.734652 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f08cb1-871e-4f39-bf35-bfe2913a6dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:12.735027 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f08cb1-871e-4f39-bf35-bfe2913a6dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:12.752233 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-88f08cb1-871e-4f39-bf35-bfe2913a6dc9 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:12.753573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-88f08cb1-871e-4f39-bf35-bfe2913a6dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:12.782439 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:12.790939 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 510/1023] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:12 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:16:12.812377 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c120ef-bfdc-49fa-8201-2a6141033a87 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:12.842192 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 514/1024] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:12 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:16:12.866211 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f08cb1-871e-4f39-bf35-bfe2913a6dc9 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:12.933941 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c120ef-bfdc-49fa-8201-2a6141033a87 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:12.964050 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:12.969929 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670c28a-4131-443d-924e-5ff3f0403c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:12.970815 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670c28a-4131-443d-924e-5ff3f0403c3d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:12.971124 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670c28a-4131-443d-924e-5ff3f0403c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:12.971401 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670c28a-4131-443d-924e-5ff3f0403c3d None admin] PATH_INFO: `/v3/services/image` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:12.972062 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f670c28a-4131-443d-924e-5ff3f0403c3d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:12.973453 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f670c28a-4131-443d-924e-5ff3f0403c3d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:12.977573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f08cb1-871e-4f39-bf35-bfe2913a6dc9 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:12.985181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-f670c28a-4131-443d-924e-5ff3f0403c3d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:16:12.985181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:12.985181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:16:12.985181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:16:12.985181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:16:12.985181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:16:12.985181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:16:12.985181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:16:12.985181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:16:12.985181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:12.985181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:16:12.985181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:12.985181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:12.985181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:12.985181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:12.985181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:12.985181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:12.986240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:12.986240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:12.986240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:12.986240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:12.986240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:12.986240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:12.986240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:16:12.986240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:16:12.986240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:16:12.986240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:16:12.986240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:12.986240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:12.986240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:12.986240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:12.986240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:12.986240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:12.987128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:12.987128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:16:12.987128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:12.987128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:12.987128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:12.987128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:12.987128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:12.987128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:12.987128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:12.987128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:12.987128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:16:12.987128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:16:12.987128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:16:12.987128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:12.988847 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 511/1025] 10.222.0.24 () {56 vars in 1105 bytes} [Fri May 27 03:16:12 2022] GET /identity/v3/services/image => generated 86 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:16:12.992379 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670c28a-4131-443d-924e-5ff3f0403c3d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:13.017432 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716e374-944f-42c2-8a9d-d6f68086ba00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:13.018253 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716e374-944f-42c2-8a9d-d6f68086ba00 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:13.018771 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716e374-944f-42c2-8a9d-d6f68086ba00 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:13.018771 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716e374-944f-42c2-8a9d-d6f68086ba00 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:13.030169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9716e374-944f-42c2-8a9d-d6f68086ba00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:16:13.039747 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-9716e374-944f-42c2-8a9d-d6f68086ba00 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:16:13.039747 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:13.039747 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:13.039747 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:13.039747 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:13.039747 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:13.039747 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:13.039747 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:13.039747 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:13.039747 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:13.039747 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:13.039747 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:13.039747 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:16:13.039747 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:16:13.039747 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:16:13.039747 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:16:13.040696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:16:13.040696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:16:13.040696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:16:13.040696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:16:13.040696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:13.040696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:13.040696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:13.040696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:13.040696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:13.040696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:13.040696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:13.040696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:16:13.040696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:13.040696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:13.040696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:13.040696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:13.041570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:13.041570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:13.041570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:13.041570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:13.041570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:16:13.041570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:16:13.041570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:16:13.041570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:16:13.041570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:16:13.041570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:16:13.041570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:16:13.041570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:16:13.041570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 515/1026] 10.222.0.24 () {56 vars in 1120 bytes} [Fri May 27 03:16:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:16:13.047054 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716e374-944f-42c2-8a9d-d6f68086ba00 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:13.058459 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:13.109484 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670c28a-4131-443d-924e-5ff3f0403c3d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:13.144072 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a530d-e609-4a2c-b836-ba243c050852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:13.145062 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a530d-e609-4a2c-b836-ba243c050852 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:13.145618 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a530d-e609-4a2c-b836-ba243c050852 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:13.145892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a530d-e609-4a2c-b836-ba243c050852 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:13.146599 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a530d-e609-4a2c-b836-ba243c050852 None admin] RBAC: Adding query filter params (name=image) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:13.146990 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a530d-e609-4a2c-b836-ba243c050852 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:13.148866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a530d-e609-4a2c-b836-ba243c050852 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:13.153609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716e374-944f-42c2-8a9d-d6f68086ba00 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:13.154062 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 512/1027] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:16:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:16:13.159620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a530d-e609-4a2c-b836-ba243c050852 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:13.190041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b7ef6-04e0-4c5a-8dcd-967e3eb5ab0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:13.191229 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b7ef6-04e0-4c5a-8dcd-967e3eb5ab0b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:13.191615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b7ef6-04e0-4c5a-8dcd-967e3eb5ab0b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:13.191894 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b7ef6-04e0-4c5a-8dcd-967e3eb5ab0b None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:13.192643 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b7ef6-04e0-4c5a-8dcd-967e3eb5ab0b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:13.192959 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b7ef6-04e0-4c5a-8dcd-967e3eb5ab0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:13.195090 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b7ef6-04e0-4c5a-8dcd-967e3eb5ab0b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:13.208602 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 516/1028] 10.222.0.24 () {56 vars in 1115 bytes} [Fri May 27 03:16:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:16:13.209289 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b7ef6-04e0-4c5a-8dcd-967e3eb5ab0b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:13.222297 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:13.266242 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a530d-e609-4a2c-b836-ba243c050852 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:13.291966 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:13.301753 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d0c05-928f-4cca-8436-ed68f7d84127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:13.302416 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d0c05-928f-4cca-8436-ed68f7d84127 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:13.302697 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d0c05-928f-4cca-8436-ed68f7d84127 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:13.305646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d0c05-928f-4cca-8436-ed68f7d84127 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:13.305646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01d0c05-928f-4cca-8436-ed68f7d84127 None admin] RBAC: Adding query filter params (type=image) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:13.305646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01d0c05-928f-4cca-8436-ed68f7d84127 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:13.305990 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01d0c05-928f-4cca-8436-ed68f7d84127 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:13.311211 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 513/1029] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:16:13 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:16:13.316792 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d0c05-928f-4cca-8436-ed68f7d84127 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:13.321903 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b7ef6-04e0-4c5a-8dcd-967e3eb5ab0b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:13.354617 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbcf4b4-27e2-4cb0-8d99-7548f9bc919b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:13.355422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbcf4b4-27e2-4cb0-8d99-7548f9bc919b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:13.355699 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbcf4b4-27e2-4cb0-8d99-7548f9bc919b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:13.355968 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbcf4b4-27e2-4cb0-8d99-7548f9bc919b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:13.367904 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbcf4b4-27e2-4cb0-8d99-7548f9bc919b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:13.369347 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbcf4b4-27e2-4cb0-8d99-7548f9bc919b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:13.386546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-acbcf4b4-27e2-4cb0-8d99-7548f9bc919b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:16:13.386546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:13.386546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:13.386546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:13.386546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:13.386546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:13.386546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:13.386546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:13.386546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:13.386546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:13.386546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:13.386546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:13.386546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:16:13.386546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:16:13.386546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:16:13.387640 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:16:13.387640 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:13.387640 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:13.387640 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:13.387640 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:13.387640 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:13.387640 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:13.387640 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:13.387640 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:16:13.387640 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:13.387640 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:13.387640 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:13.387640 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:13.387640 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:13.387640 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:13.387640 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:13.387640 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:13.388546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:16:13.388546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:16:13.388546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:16:13.388546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:16:13.388546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 517/1030] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:16:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:16:13.393199 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbcf4b4-27e2-4cb0-8d99-7548f9bc919b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:13.426923 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d0c05-928f-4cca-8436-ed68f7d84127 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:13.458692 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:13.463717 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e694d3f-3d8f-4ed6-acdb-99d0af58e20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:13.464380 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e694d3f-3d8f-4ed6-acdb-99d0af58e20a None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:13.465236 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e694d3f-3d8f-4ed6-acdb-99d0af58e20a None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:13.465236 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e694d3f-3d8f-4ed6-acdb-99d0af58e20a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:13.465801 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e694d3f-3d8f-4ed6-acdb-99d0af58e20a None admin] RBAC: Adding query filter params (interface=public, service_id=46ea1f151a664154b0b740e592bf179a, region_id=RegionOne) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:13.466043 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e694d3f-3d8f-4ed6-acdb-99d0af58e20a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:13.467436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e694d3f-3d8f-4ed6-acdb-99d0af58e20a None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:13.477243 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 514/1031] 10.222.0.24 () {56 vars in 1251 bytes} [Fri May 27 03:16:13 2022] GET /identity/v3/endpoints?service_id=46ea1f151a664154b0b740e592bf179a&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:16:13.478139 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e694d3f-3d8f-4ed6-acdb-99d0af58e20a None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:13.504093 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbcf4b4-27e2-4cb0-8d99-7548f9bc919b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:13.539387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f3e72-e8dd-4dc8-9075-93438eb49ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:13.539715 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:13.540049 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f3e72-e8dd-4dc8-9075-93438eb49ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:13.540327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f3e72-e8dd-4dc8-9075-93438eb49ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:13.540600 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f3e72-e8dd-4dc8-9075-93438eb49ac4 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:13.541319 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22f3e72-e8dd-4dc8-9075-93438eb49ac4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:13.541623 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22f3e72-e8dd-4dc8-9075-93438eb49ac4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:13.543023 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22f3e72-e8dd-4dc8-9075-93438eb49ac4 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:13.556055 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 518/1032] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:16:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:16:13.561497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f3e72-e8dd-4dc8-9075-93438eb49ac4 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:13.588620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e694d3f-3d8f-4ed6-acdb-99d0af58e20a None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:13.622891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e47e5d-5269-4b36-b1a0-33edf57ba0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:13.623577 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e47e5d-5269-4b36-b1a0-33edf57ba0ce None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:13.623854 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e47e5d-5269-4b36-b1a0-33edf57ba0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:13.624124 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e47e5d-5269-4b36-b1a0-33edf57ba0ce None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:13.624789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e47e5d-5269-4b36-b1a0-33edf57ba0ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:13.626288 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e47e5d-5269-4b36-b1a0-33edf57ba0ce None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:13.642872 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 515/1033] 10.222.0.24 () {56 vars in 1087 bytes} [Fri May 27 03:16:13 2022] GET /identity/v3/services => generated 1957 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:16:13.682708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f3e72-e8dd-4dc8-9075-93438eb49ac4 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:13.708084 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:13.720734 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406caa77-6556-47cd-8e2c-00769570df3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:13.721509 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406caa77-6556-47cd-8e2c-00769570df3a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:13.721795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406caa77-6556-47cd-8e2c-00769570df3a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:13.722070 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406caa77-6556-47cd-8e2c-00769570df3a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:13.730958 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406caa77-6556-47cd-8e2c-00769570df3a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:13.732515 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406caa77-6556-47cd-8e2c-00769570df3a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:13.749985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-406caa77-6556-47cd-8e2c-00769570df3a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:16:13.749985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:13.749985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:13.749985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:13.749985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:13.749985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:13.749985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:13.749985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:13.749985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:13.749985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:13.749985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:13.749985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:13.749985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:16:13.749985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:16:13.749985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:16:13.749985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:16:13.756196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:13.756196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:13.756196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:16:13.756196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:16:13.756196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:16:13.756196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:16:13.756196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:13.756196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:13.756196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:13.756196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:13.756196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:13.756196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:16:13.756196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:13.756196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:13.756196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:13.756196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:13.756196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:13.756196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:13.757365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:13.757365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:13.757365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:16:13.757365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:16:13.757365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:16:13.757365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:16:13.757365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:16:13.757365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:16:13.757365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:16:13.757365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:16:13.757365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 519/1034] 10.222.0.24 () {56 vars in 1093 bytes} [Fri May 27 03:16:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:16:13.757983 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e47e5d-5269-4b36-b1a0-33edf57ba0ce None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:13.869412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e47e5d-5269-4b36-b1a0-33edf57ba0ce None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:13.887287 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:13.904536 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d4a19-da2c-403f-9558-5d832f55c493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:13.905585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d4a19-da2c-403f-9558-5d832f55c493 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:13.906106 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d4a19-da2c-403f-9558-5d832f55c493 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:13.906574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d4a19-da2c-403f-9558-5d832f55c493 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:13.907624 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d4a19-da2c-403f-9558-5d832f55c493 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:13.908132 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d4a19-da2c-403f-9558-5d832f55c493 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:13.909749 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d4a19-da2c-403f-9558-5d832f55c493 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:13.929235 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 516/1035] 10.222.0.24 () {56 vars in 1133 bytes} [Fri May 27 03:16:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:16:13.935122 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406caa77-6556-47cd-8e2c-00769570df3a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:14.035570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406caa77-6556-47cd-8e2c-00769570df3a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:14.068731 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736daeb-e82d-4e7e-8bc3-e2a448af9197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:14.069375 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736daeb-e82d-4e7e-8bc3-e2a448af9197 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:14.069653 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736daeb-e82d-4e7e-8bc3-e2a448af9197 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:14.069928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736daeb-e82d-4e7e-8bc3-e2a448af9197 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:14.123188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:14.150346 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736daeb-e82d-4e7e-8bc3-e2a448af9197 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:14.151854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736daeb-e82d-4e7e-8bc3-e2a448af9197 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:14.265291 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 520/1036] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:16:13 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:16:14.266577 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d4a19-da2c-403f-9558-5d832f55c493 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:14.366260 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d4a19-da2c-403f-9558-5d832f55c493 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:14.400648 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28779dd7-a750-4464-9ad9-588c98b188be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:14.401498 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28779dd7-a750-4464-9ad9-588c98b188be None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:14.401787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28779dd7-a750-4464-9ad9-588c98b188be None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:14.402061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28779dd7-a750-4464-9ad9-588c98b188be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:14.412414 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28779dd7-a750-4464-9ad9-588c98b188be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:14.413901 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28779dd7-a750-4464-9ad9-588c98b188be None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:14.431860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-28779dd7-a750-4464-9ad9-588c98b188be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:16:14.431860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:14.431860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:14.431860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:14.431860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:14.431860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:14.431860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:14.431860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:14.431860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:14.431860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:14.431860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:14.431860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:14.431860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:16:14.431860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:16:14.431860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:16:14.432752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:16:14.432752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:14.432752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:14.432752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:14.432752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:14.432752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:14.432752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:14.432752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:14.432752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:16:14.432752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:14.432752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:14.432752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:14.432752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:14.432752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:14.432752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:14.432752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:14.432752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:14.438274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:16:14.438274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:16:14.438274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:16:14.438274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:14.438274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 517/1037] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:16:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:16:14.438879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736daeb-e82d-4e7e-8bc3-e2a448af9197 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:14.528874 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:14.548590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736daeb-e82d-4e7e-8bc3-e2a448af9197 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:14.582873 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606868d-a5ce-49cb-89ac-eb94d0ace800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:14.583634 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606868d-a5ce-49cb-89ac-eb94d0ace800 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:14.583920 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606868d-a5ce-49cb-89ac-eb94d0ace800 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:14.584192 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606868d-a5ce-49cb-89ac-eb94d0ace800 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:14.585471 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4606868d-a5ce-49cb-89ac-eb94d0ace800 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:14.585783 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4606868d-a5ce-49cb-89ac-eb94d0ace800 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:14.587164 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4606868d-a5ce-49cb-89ac-eb94d0ace800 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:14.598642 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 521/1038] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:16:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:14.604211 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28779dd7-a750-4464-9ad9-588c98b188be None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:14.634277 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:14.724426 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28779dd7-a750-4464-9ad9-588c98b188be None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:14.768861 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861bd7e1-e4b5-449f-b187-f5d36728681b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:14.769697 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bd7e1-e4b5-449f-b187-f5d36728681b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:14.769987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bd7e1-e4b5-449f-b187-f5d36728681b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:14.770327 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bd7e1-e4b5-449f-b187-f5d36728681b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:14.778342 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861bd7e1-e4b5-449f-b187-f5d36728681b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:14.779888 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861bd7e1-e4b5-449f-b187-f5d36728681b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:14.795595 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-861bd7e1-e4b5-449f-b187-f5d36728681b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:16:14.795595 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:14.795595 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:14.795595 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:14.795595 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:14.795595 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:14.795595 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:14.795595 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:14.795595 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:14.795595 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:14.795595 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:14.795595 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:14.795595 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:16:14.795595 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:16:14.795595 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:16:14.796705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:16:14.796705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:14.796705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:14.796705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:14.796705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:14.796705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:14.796705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:14.796705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:14.796705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:16:14.796705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:14.796705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:14.796705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:14.796705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:14.796705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:14.796705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:14.796705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:14.796705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:14.805175 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:16:14.805175 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:16:14.805175 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:16:14.805175 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:16:14.805175 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:16:14.805175 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:16:14.805175 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:16:14.805175 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:14.805175 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 518/1039] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:16:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:16:14.807556 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606868d-a5ce-49cb-89ac-eb94d0ace800 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:14.926402 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606868d-a5ce-49cb-89ac-eb94d0ace800 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:14.948370 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41470c-7920-4ac3-aaa0-7c10cde13ff0 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:14.949307 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41470c-7920-4ac3-aaa0-7c10cde13ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:14.949931 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c41470c-7920-4ac3-aaa0-7c10cde13ff0 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:14.951539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 519/1040] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:16:14.957197 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50baecb-a793-434d-b700-5594cb9286cb None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:14.957479 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50baecb-a793-434d-b700-5594cb9286cb None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:14.957859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50baecb-a793-434d-b700-5594cb9286cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:14.964830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee99edc-1f11-41c1-979d-b8863886add4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:14.965675 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee99edc-1f11-41c1-979d-b8863886add4 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:14.965964 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee99edc-1f11-41c1-979d-b8863886add4 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:14.967089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee99edc-1f11-41c1-979d-b8863886add4 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:14.967927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee99edc-1f11-41c1-979d-b8863886add4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:14.968246 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee99edc-1f11-41c1-979d-b8863886add4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:14.969698 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee99edc-1f11-41c1-979d-b8863886add4 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:14.977325 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-e50baecb-a793-434d-b700-5594cb9286cb None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:14.978505 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e50baecb-a793-434d-b700-5594cb9286cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:14.982187 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 522/1041] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:16:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:14.990014 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee99edc-1f11-41c1-979d-b8863886add4 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:14.993825 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:15.058232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 520/1042] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:14 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:16:15.084402 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50baecb-a793-434d-b700-5594cb9286cb None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:15.104017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:15.118829 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee99edc-1f11-41c1-979d-b8863886add4 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:15.157412 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aea412-ed6b-4646-9662-3f3c87111e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:15.158079 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aea412-ed6b-4646-9662-3f3c87111e9f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:15.158360 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aea412-ed6b-4646-9662-3f3c87111e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:15.158638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aea412-ed6b-4646-9662-3f3c87111e9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:15.159507 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aea412-ed6b-4646-9662-3f3c87111e9f None admin] RBAC: Adding query filter params (role.id=1705f116742c4217bb8f6eab32b40850, scope.project.id=6ceffdd955324aec84ba048d2137ae00, user.id=22a9592143014f3287cd75c60ca17237) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:15.159822 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aea412-ed6b-4646-9662-3f3c87111e9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:15.161262 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aea412-ed6b-4646-9662-3f3c87111e9f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:15.168440 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 523/1043] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:16:14 2022] GET /identity/v3/role_assignments?user.id=22a9592143014f3287cd75c60ca17237&scope.project.id=6ceffdd955324aec84ba048d2137ae00&role.id=1705f116742c4217bb8f6eab32b40850 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:16:15.204477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50baecb-a793-434d-b700-5594cb9286cb None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:15.246210 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7376d40-cb53-43ad-828e-77f1febcf705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:15.247066 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7376d40-cb53-43ad-828e-77f1febcf705 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:15.247355 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7376d40-cb53-43ad-828e-77f1febcf705 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:15.247627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7376d40-cb53-43ad-828e-77f1febcf705 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:15.248885 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7376d40-cb53-43ad-828e-77f1febcf705 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:15.250396 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7376d40-cb53-43ad-828e-77f1febcf705 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:15.252517 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 521/1044] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:16:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:16:15.259940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aea412-ed6b-4646-9662-3f3c87111e9f None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:15.332965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:15.372990 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aea412-ed6b-4646-9662-3f3c87111e9f None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:15.406566 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1df52-4ab2-4b2c-90d6-827c2e39cd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:15.407275 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1df52-4ab2-4b2c-90d6-827c2e39cd67 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:15.407556 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1df52-4ab2-4b2c-90d6-827c2e39cd67 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:15.407825 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1df52-4ab2-4b2c-90d6-827c2e39cd67 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:15.408540 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c1df52-4ab2-4b2c-90d6-827c2e39cd67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:15.410055 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c1df52-4ab2-4b2c-90d6-827c2e39cd67 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:15.510461 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 524/1045] 10.222.0.24 () {60 vars in 1132 bytes} [Fri May 27 03:16:15 2022] POST /identity/v3/users => generated 285 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:16:15.899999 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181348da-9ffa-4c5d-a3db-d5a2d20a7112 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:15.900523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181348da-9ffa-4c5d-a3db-d5a2d20a7112 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:15.900523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181348da-9ffa-4c5d-a3db-d5a2d20a7112 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:15.902052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 522/1046] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:15.907421 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22879dfa-1513-4243-9114-c89179be488c None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:15.907718 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22879dfa-1513-4243-9114-c89179be488c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:15.907996 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22879dfa-1513-4243-9114-c89179be488c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:15.924939 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-22879dfa-1513-4243-9114-c89179be488c None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:15.926117 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-22879dfa-1513-4243-9114-c89179be488c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:15.994603 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 525/1047] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:15 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:16:16.017001 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181348da-9ffa-4c5d-a3db-d5a2d20a7112 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:16.111749 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:16.126927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181348da-9ffa-4c5d-a3db-d5a2d20a7112 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:16.161198 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6b55a-5c4f-4899-9f35-1f0bfff31aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:16.161911 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6b55a-5c4f-4899-9f35-1f0bfff31aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:16.162191 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6b55a-5c4f-4899-9f35-1f0bfff31aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:16.162496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6b55a-5c4f-4899-9f35-1f0bfff31aa7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:16.163177 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f6b55a-5c4f-4899-9f35-1f0bfff31aa7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:16.164535 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f6b55a-5c4f-4899-9f35-1f0bfff31aa7 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:16.175901 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-f8f6b55a-5c4f-4899-9f35-1f0bfff31aa7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:16:16.175901 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:16.175901 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:16:16.175901 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:16:16.175901 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:16:16.175901 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:16:16.175901 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:16:16.175901 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:16:16.175901 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:16:16.175901 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:16.175901 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:16:16.175901 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:16.175901 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:16.175901 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:16.175901 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:16.175901 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:16.175901 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:16.176877 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:16.176877 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:16.176877 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:16.176877 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:16.176877 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:16.176877 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:16.176877 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:16:16.176877 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:16:16.176877 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:16:16.176877 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:16:16.176877 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:16.176877 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:16.176877 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:16.176877 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:16.176877 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:16.176877 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:16.183251 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:16.183251 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:16:16.183251 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:16.183251 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:16.183251 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:16.183251 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:16.183251 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:16.183251 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:16.183251 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:16.183251 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:16.183251 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:16:16.183251 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:16:16.183251 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:16:16.183251 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:16.183251 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 523/1048] 10.222.0.24 () {56 vars in 1105 bytes} [Fri May 27 03:16:16 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:16:16.184030 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22879dfa-1513-4243-9114-c89179be488c None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:16.235059 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:16.283339 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22879dfa-1513-4243-9114-c89179be488c None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:16.319524 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973d890-cce8-43c9-a265-7ee1479a8803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:16.320195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973d890-cce8-43c9-a265-7ee1479a8803 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:16.320485 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973d890-cce8-43c9-a265-7ee1479a8803 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:16.320776 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973d890-cce8-43c9-a265-7ee1479a8803 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:16.321441 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973d890-cce8-43c9-a265-7ee1479a8803 None admin] RBAC: Adding query filter params (name=image) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:16.321745 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973d890-cce8-43c9-a265-7ee1479a8803 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:16.323138 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973d890-cce8-43c9-a265-7ee1479a8803 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:16.333117 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 526/1049] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:16:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:16:16.333915 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6b55a-5c4f-4899-9f35-1f0bfff31aa7 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:16.434237 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6b55a-5c4f-4899-9f35-1f0bfff31aa7 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:16.467389 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a8141-9af0-4819-9178-31284a4f80e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:16.468054 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a8141-9af0-4819-9178-31284a4f80e9 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:16.468335 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a8141-9af0-4819-9178-31284a4f80e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:16.468610 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a8141-9af0-4819-9178-31284a4f80e9 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:16.469310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a8141-9af0-4819-9178-31284a4f80e9 None admin] RBAC: Adding query filter params (type=image) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:16.469614 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a8141-9af0-4819-9178-31284a4f80e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:16.471012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539a8141-9af0-4819-9178-31284a4f80e9 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:16.480327 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 524/1050] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:16:16 2022] GET /identity/v3/services?type=image => generated 350 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:16:16.481853 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973d890-cce8-43c9-a265-7ee1479a8803 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:16.589312 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973d890-cce8-43c9-a265-7ee1479a8803 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:16.605803 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:16.624205 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d199b77-892f-4249-8d38-23986a41f5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:16.624980 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d199b77-892f-4249-8d38-23986a41f5b3 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:16.625268 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d199b77-892f-4249-8d38-23986a41f5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:16.625544 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d199b77-892f-4249-8d38-23986a41f5b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:16.626295 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d199b77-892f-4249-8d38-23986a41f5b3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:16.627682 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d199b77-892f-4249-8d38-23986a41f5b3 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:16.745586 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 527/1051] 10.222.0.24 () {60 vars in 1144 bytes} [Fri May 27 03:16:16 2022] POST /identity/v3/endpoints => generated 327 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:16:17.397244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6262939-d085-481e-8db3-ee35308d3a80 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:17.397760 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6262939-d085-481e-8db3-ee35308d3a80 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:17.397760 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6262939-d085-481e-8db3-ee35308d3a80 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:17.399255 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 525/1052] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:17.404239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466f984-91c0-4f22-beff-68cf12a171e9 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:17.404629 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466f984-91c0-4f22-beff-68cf12a171e9 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:17.404807 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466f984-91c0-4f22-beff-68cf12a171e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:17.421914 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-8466f984-91c0-4f22-beff-68cf12a171e9 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:17.423119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-8466f984-91c0-4f22-beff-68cf12a171e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:17.495894 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 528/1053] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:16:17.518558 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6262939-d085-481e-8db3-ee35308d3a80 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:17.535180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:17.624263 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6262939-d085-481e-8db3-ee35308d3a80 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:17.659594 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2515d68d-9d6d-42c8-a248-177f43451031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:17.660320 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515d68d-9d6d-42c8-a248-177f43451031 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:17.660601 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515d68d-9d6d-42c8-a248-177f43451031 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:17.660894 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515d68d-9d6d-42c8-a248-177f43451031 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:17.671444 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2515d68d-9d6d-42c8-a248-177f43451031 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:16:17.680137 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-2515d68d-9d6d-42c8-a248-177f43451031 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:16:17.680137 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:17.680137 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:17.680137 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:17.680137 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:17.680137 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:17.680137 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:17.680137 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:17.680137 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:17.680137 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:17.680137 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:17.680137 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:17.680137 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:16:17.680137 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:16:17.680137 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:16:17.680137 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:16:17.681127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:16:17.681127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:16:17.681127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:16:17.681127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:16:17.681127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:17.681127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:17.681127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:17.681127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:17.681127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:17.681127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:17.681127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:17.681127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:16:17.681127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:17.681127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:17.681127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:17.681127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:17.681959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:17.681959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:17.681959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:17.681959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:17.681959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:16:17.681959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:16:17.681959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:16:17.681959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:16:17.681959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:16:17.681959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:16:17.681959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:16:17.681959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:17.683482 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 526/1054] 10.222.0.24 () {56 vars in 1120 bytes} [Fri May 27 03:16:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:16:17.687362 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466f984-91c0-4f22-beff-68cf12a171e9 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:17.781960 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:17.785985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466f984-91c0-4f22-beff-68cf12a171e9 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:17.795380 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f4e7e-5ed4-48ca-8802-5cd5fe139832 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:17.795665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f4e7e-5ed4-48ca-8802-5cd5fe139832 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:17.795954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f4e7e-5ed4-48ca-8802-5cd5fe139832 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:17.797422 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 527/1055] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:16:17.802078 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53632bdc-bf5b-4905-9b61-7f5dbddc1f5e None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:17.802362 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53632bdc-bf5b-4905-9b61-7f5dbddc1f5e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:17.802635 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53632bdc-bf5b-4905-9b61-7f5dbddc1f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:17.819235 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-53632bdc-bf5b-4905-9b61-7f5dbddc1f5e None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:17.819839 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0a9c5-e95f-434e-aaa9-a18dbf75616a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:17.820392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-53632bdc-bf5b-4905-9b61-7f5dbddc1f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:17.820504 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0a9c5-e95f-434e-aaa9-a18dbf75616a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:17.820777 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0a9c5-e95f-434e-aaa9-a18dbf75616a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:17.821051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0a9c5-e95f-434e-aaa9-a18dbf75616a None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:17.821793 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0a9c5-e95f-434e-aaa9-a18dbf75616a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:17.822099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0a9c5-e95f-434e-aaa9-a18dbf75616a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:17.823516 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0a9c5-e95f-434e-aaa9-a18dbf75616a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:17.832572 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 529/1056] 10.222.0.24 () {56 vars in 1115 bytes} [Fri May 27 03:16:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:17.838741 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0a9c5-e95f-434e-aaa9-a18dbf75616a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:17.897424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 528/1057] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:16:17.920791 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53632bdc-bf5b-4905-9b61-7f5dbddc1f5e None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:17.924278 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:17.953020 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0a9c5-e95f-434e-aaa9-a18dbf75616a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:17.988737 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12330a94-718f-4375-a997-ae1a4958f0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:17.989523 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12330a94-718f-4375-a997-ae1a4958f0f1 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:17.989800 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12330a94-718f-4375-a997-ae1a4958f0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:17.990066 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12330a94-718f-4375-a997-ae1a4958f0f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:18.000830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12330a94-718f-4375-a997-ae1a4958f0f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:18.002270 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12330a94-718f-4375-a997-ae1a4958f0f1 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:18.019191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-12330a94-718f-4375-a997-ae1a4958f0f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:16:18.019191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:18.019191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:18.019191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:18.019191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:18.019191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:18.019191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:18.019191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:18.019191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:18.019191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:18.019191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:18.019191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:18.019191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:16:18.019191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:16:18.019191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:16:18.020297 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:16:18.020297 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:18.020297 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:18.020297 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:18.020297 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:18.020297 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:18.020297 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:18.020297 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:18.020297 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:16:18.020297 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:18.020297 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:18.020297 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:18.020297 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:18.020297 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:18.020297 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:18.020297 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:18.020297 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:18.021405 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:16:18.021405 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:16:18.021405 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:16:18.021405 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:16:18.025787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 530/1058] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:16:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:16:18.026502 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12330a94-718f-4375-a997-ae1a4958f0f1 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:18.030944 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53632bdc-bf5b-4905-9b61-7f5dbddc1f5e None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:18.043458 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:18.066017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8906a3b-4308-4ac0-bd1d-4547019c2623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:18.066809 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8906a3b-4308-4ac0-bd1d-4547019c2623 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:18.067114 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8906a3b-4308-4ac0-bd1d-4547019c2623 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:18.067389 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8906a3b-4308-4ac0-bd1d-4547019c2623 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:18.083099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8906a3b-4308-4ac0-bd1d-4547019c2623 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:16:18.091393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-b8906a3b-4308-4ac0-bd1d-4547019c2623 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:16:18.091393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:18.091393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:18.091393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:18.091393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:18.091393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:18.091393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:18.091393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:18.091393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:18.091393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:18.091393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:18.091393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:18.091393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:16:18.091393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:16:18.091393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:16:18.091393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:16:18.092315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:16:18.092315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:16:18.092315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:16:18.092315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:16:18.092315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:18.092315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:18.092315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:18.092315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:18.092315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:18.092315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:18.092315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:18.092315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:16:18.092315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:18.092315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:18.092315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:18.092315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:18.093724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:18.093724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:18.093724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:18.093724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:18.093724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:16:18.093724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:16:18.093724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:16:18.093724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:16:18.093724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:16:18.093724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:16:18.093724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:16:18.093724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:18.093724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 529/1059] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:16:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:16:18.098347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8906a3b-4308-4ac0-bd1d-4547019c2623 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:18.143761 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12330a94-718f-4375-a997-ae1a4958f0f1 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:18.164691 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:18.179852 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184f687-ccb6-4bee-a3a4-cc9cbb7e46bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:18.180438 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184f687-ccb6-4bee-a3a4-cc9cbb7e46bc None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:18.180719 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184f687-ccb6-4bee-a3a4-cc9cbb7e46bc None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:18.180992 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184f687-ccb6-4bee-a3a4-cc9cbb7e46bc None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:18.181858 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0184f687-ccb6-4bee-a3a4-cc9cbb7e46bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:18.182009 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0184f687-ccb6-4bee-a3a4-cc9cbb7e46bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:18.183476 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0184f687-ccb6-4bee-a3a4-cc9cbb7e46bc None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:18.196189 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 531/1060] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:18.201743 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184f687-ccb6-4bee-a3a4-cc9cbb7e46bc None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:18.205045 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8906a3b-4308-4ac0-bd1d-4547019c2623 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:18.239577 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793d4c2-a200-475c-8f11-b477bb4b9742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:18.240241 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793d4c2-a200-475c-8f11-b477bb4b9742 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:18.240549 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793d4c2-a200-475c-8f11-b477bb4b9742 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:18.240827 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793d4c2-a200-475c-8f11-b477bb4b9742 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:18.241573 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4793d4c2-a200-475c-8f11-b477bb4b9742 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:18.241882 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4793d4c2-a200-475c-8f11-b477bb4b9742 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:18.243279 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4793d4c2-a200-475c-8f11-b477bb4b9742 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:18.256845 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 530/1061] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:16:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:16:18.257931 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793d4c2-a200-475c-8f11-b477bb4b9742 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:18.280032 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:18.314478 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0184f687-ccb6-4bee-a3a4-cc9cbb7e46bc None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:18.356296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d5dc7-00aa-42fd-a91c-6418b3257271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:18.357094 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d5dc7-00aa-42fd-a91c-6418b3257271 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:18.357372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d5dc7-00aa-42fd-a91c-6418b3257271 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:18.357652 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d5dc7-00aa-42fd-a91c-6418b3257271 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:18.362466 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793d4c2-a200-475c-8f11-b477bb4b9742 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:18.366253 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d5dc7-00aa-42fd-a91c-6418b3257271 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:18.367726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d5dc7-00aa-42fd-a91c-6418b3257271 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:18.383251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-597d5dc7-00aa-42fd-a91c-6418b3257271 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:16:18.383251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:18.383251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:18.383251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:18.383251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:18.383251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:18.383251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:18.383251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:18.383251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:18.383251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:18.383251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:18.383251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:18.383251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:16:18.383251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:16:18.383251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:16:18.383251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:16:18.384228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:18.384228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:18.384228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:16:18.384228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:16:18.384228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:16:18.384228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:16:18.384228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:18.384228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:18.384228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:18.384228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:18.384228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:18.384228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:16:18.384228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:18.384228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:18.384228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:18.384228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:18.384228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:18.384228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:18.385209 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:18.385209 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:18.385209 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:16:18.385209 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:16:18.385209 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:16:18.385209 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:16:18.385209 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:16:18.385209 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:16:18.385209 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:16:18.385209 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:16:18.385209 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 532/1062] 10.222.0.24 () {56 vars in 1093 bytes} [Fri May 27 03:16:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:16:18.390127 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d5dc7-00aa-42fd-a91c-6418b3257271 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:18.391066 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:18.400122 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c329b4d1-a45b-4083-8ce8-2805372ef961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:18.403959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329b4d1-a45b-4083-8ce8-2805372ef961 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:18.403959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329b4d1-a45b-4083-8ce8-2805372ef961 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:18.403959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329b4d1-a45b-4083-8ce8-2805372ef961 None admin] PATH_INFO: `/v3/users/d9a554766bf649a2b5b3bffa155bc3da` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:18.415565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c329b4d1-a45b-4083-8ce8-2805372ef961 None admin] RBAC: Authorizing `identity:get_user(user_id=d9a554766bf649a2b5b3bffa155bc3da)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:18.416997 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c329b4d1-a45b-4083-8ce8-2805372ef961 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:18.419010 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 531/1063] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:16:18 2022] GET /identity/v3/users/d9a554766bf649a2b5b3bffa155bc3da => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:16:18.424982 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c329b4d1-a45b-4083-8ce8-2805372ef961 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:18.499725 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d5dc7-00aa-42fd-a91c-6418b3257271 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:18.531943 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:18.534278 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c329b4d1-a45b-4083-8ce8-2805372ef961 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:18.539132 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623eb60-8643-4bdd-bca0-36bb1a73185c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:18.539813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623eb60-8643-4bdd-bca0-36bb1a73185c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:18.540148 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623eb60-8643-4bdd-bca0-36bb1a73185c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:18.540431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4623eb60-8643-4bdd-bca0-36bb1a73185c None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:18.541276 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4623eb60-8643-4bdd-bca0-36bb1a73185c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:18.550042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4623eb60-8643-4bdd-bca0-36bb1a73185c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:18.550042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4623eb60-8643-4bdd-bca0-36bb1a73185c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:18.562689 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 533/1064] 10.222.0.24 () {56 vars in 1133 bytes} [Fri May 27 03:16:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:16:18.568309 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623eb60-8643-4bdd-bca0-36bb1a73185c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:18.571221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801b2101-32ee-4fe9-85e5-c316ea35949c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:18.571878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b2101-32ee-4fe9-85e5-c316ea35949c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:18.572162 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b2101-32ee-4fe9-85e5-c316ea35949c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:18.573061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801b2101-32ee-4fe9-85e5-c316ea35949c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:18.629887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:18.648158 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801b2101-32ee-4fe9-85e5-c316ea35949c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:18.652724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801b2101-32ee-4fe9-85e5-c316ea35949c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:18.676921 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4623eb60-8643-4bdd-bca0-36bb1a73185c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:18.713295 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668afa0-f63e-49b3-828a-20190354853c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:18.714083 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668afa0-f63e-49b3-828a-20190354853c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:18.714359 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668afa0-f63e-49b3-828a-20190354853c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:18.714630 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668afa0-f63e-49b3-828a-20190354853c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:18.725609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b668afa0-f63e-49b3-828a-20190354853c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:18.727081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b668afa0-f63e-49b3-828a-20190354853c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:18.745659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-b668afa0-f63e-49b3-828a-20190354853c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:16:18.745659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:18.745659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:18.745659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:18.745659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:18.745659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:18.745659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:18.745659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:18.745659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:18.745659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:18.745659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:18.745659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:18.745659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:16:18.745659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:16:18.745659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:16:18.746590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:16:18.746590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:18.746590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:18.746590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:18.746590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:18.746590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:18.746590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:18.746590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:18.746590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:16:18.746590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:18.746590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:18.746590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:18.746590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:18.746590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:18.746590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:18.746590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:18.746590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:18.747707 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:16:18.747707 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:16:18.747707 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:16:18.747707 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:16:18.747707 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 534/1065] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:16:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:16:18.752327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668afa0-f63e-49b3-828a-20190354853c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:18.759095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 532/1066] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:16:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 03:16:18.764353 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801b2101-32ee-4fe9-85e5-c316ea35949c None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:18.800429 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:18.856445 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:18.868126 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668afa0-f63e-49b3-828a-20190354853c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:18.876016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801b2101-32ee-4fe9-85e5-c316ea35949c None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:18.908790 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ba6e9-c878-4375-a894-fa5ed5c1058f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:18.909459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ba6e9-c878-4375-a894-fa5ed5c1058f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:18.909739 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ba6e9-c878-4375-a894-fa5ed5c1058f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:18.910008 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ba6e9-c878-4375-a894-fa5ed5c1058f None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:18.910853 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026ba6e9-c878-4375-a894-fa5ed5c1058f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:18.911204 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026ba6e9-c878-4375-a894-fa5ed5c1058f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:18.914108 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026ba6e9-c878-4375-a894-fa5ed5c1058f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:18.917107 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28c71c-df8c-43cd-8ebe-a7aa8206b34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:18.917970 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28c71c-df8c-43cd-8ebe-a7aa8206b34c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:18.918254 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28c71c-df8c-43cd-8ebe-a7aa8206b34c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:18.918535 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28c71c-df8c-43cd-8ebe-a7aa8206b34c None admin] PATH_INFO: `/v3/projects/01eb74db1c404662890e270313372e72` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:18.919763 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28c71c-df8c-43cd-8ebe-a7aa8206b34c None admin] RBAC: Authorizing `identity:get_project(project_id=01eb74db1c404662890e270313372e72)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:18.921192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28c71c-df8c-43cd-8ebe-a7aa8206b34c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:18.923070 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 533/1067] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:16:18 2022] GET /identity/v3/projects/01eb74db1c404662890e270313372e72 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:16:18.926216 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 535/1068] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:16:18.928878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28c71c-df8c-43cd-8ebe-a7aa8206b34c None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:18.931789 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ba6e9-c878-4375-a894-fa5ed5c1058f None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:19.029704 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28c71c-df8c-43cd-8ebe-a7aa8206b34c None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:19.059631 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ba6e9-c878-4375-a894-fa5ed5c1058f None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:19.061126 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:19.061710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47409c-a5f0-48b7-8cf9-8de7d406e4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:19.062367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47409c-a5f0-48b7-8cf9-8de7d406e4de None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:19.062646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47409c-a5f0-48b7-8cf9-8de7d406e4de None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:19.062946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47409c-a5f0-48b7-8cf9-8de7d406e4de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:19.063765 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47409c-a5f0-48b7-8cf9-8de7d406e4de None admin] RBAC: Adding query filter params (role.id=e5a908bd3ed5445b82a6053246533b88, scope.project.id=01eb74db1c404662890e270313372e72, user.id=d9a554766bf649a2b5b3bffa155bc3da) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:19.064074 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47409c-a5f0-48b7-8cf9-8de7d406e4de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:19.065499 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47409c-a5f0-48b7-8cf9-8de7d406e4de None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:19.071482 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 534/1069] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:16:18 2022] GET /identity/v3/role_assignments?user.id=d9a554766bf649a2b5b3bffa155bc3da&scope.project.id=01eb74db1c404662890e270313372e72&role.id=e5a908bd3ed5445b82a6053246533b88 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:16:19.072798 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae664814-aafa-49d0-acab-dec3851abb4c None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:19.073084 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae664814-aafa-49d0-acab-dec3851abb4c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:19.073357 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae664814-aafa-49d0-acab-dec3851abb4c None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:19.075047 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 535/1070] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:19.080759 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1479056-3150-4d65-94af-8726d1815ee7 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:19.081054 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1479056-3150-4d65-94af-8726d1815ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:19.081331 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1479056-3150-4d65-94af-8726d1815ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:19.101944 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1e23c-a038-4e23-895f-4e1bb97b55fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:19.102759 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1e23c-a038-4e23-895f-4e1bb97b55fe None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:19.103096 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1e23c-a038-4e23-895f-4e1bb97b55fe None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:19.103374 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1e23c-a038-4e23-895f-4e1bb97b55fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:19.105977 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-f1479056-3150-4d65-94af-8726d1815ee7 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:19.107682 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f1479056-3150-4d65-94af-8726d1815ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:19.111128 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a1e23c-a038-4e23-895f-4e1bb97b55fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:19.112604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a1e23c-a038-4e23-895f-4e1bb97b55fe None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:19.120721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:19.127949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-90a1e23c-a038-4e23-895f-4e1bb97b55fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:16:19.127949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:19.127949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:19.127949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:19.127949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:19.127949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:19.127949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:19.127949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:19.127949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:19.127949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:19.127949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:19.127949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:19.127949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:16:19.127949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:16:19.127949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:16:19.129089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:16:19.129089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:19.129089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:19.129089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:19.129089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:19.129089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:19.129089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:19.129089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:19.129089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:16:19.129089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:19.129089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:19.129089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:19.129089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:19.129089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:19.129089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:19.129089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:19.129089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:19.130818 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:16:19.130818 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:16:19.130818 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:16:19.130818 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:16:19.130818 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:16:19.130818 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:16:19.130818 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:16:19.130818 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:16:19.130818 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 536/1071] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:16:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:16:19.137541 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1e23c-a038-4e23-895f-4e1bb97b55fe None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:19.187997 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 536/1072] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:16:19.212134 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1479056-3150-4d65-94af-8726d1815ee7 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:19.258190 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1e23c-a038-4e23-895f-4e1bb97b55fe None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:19.293715 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676c608-e281-437b-b3dd-4ec4cc8ff10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:19.294382 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676c608-e281-437b-b3dd-4ec4cc8ff10f None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:19.294663 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676c608-e281-437b-b3dd-4ec4cc8ff10f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:19.294971 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676c608-e281-437b-b3dd-4ec4cc8ff10f None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:19.295761 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a676c608-e281-437b-b3dd-4ec4cc8ff10f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:19.296070 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a676c608-e281-437b-b3dd-4ec4cc8ff10f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:19.297495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a676c608-e281-437b-b3dd-4ec4cc8ff10f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:19.314994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 537/1073] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:16:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:19.315877 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676c608-e281-437b-b3dd-4ec4cc8ff10f None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:19.333470 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1479056-3150-4d65-94af-8726d1815ee7 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:19.340503 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:19.370095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db205745-247e-4fdf-b65a-83f0f2520a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:19.370746 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db205745-247e-4fdf-b65a-83f0f2520a90 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:19.371104 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db205745-247e-4fdf-b65a-83f0f2520a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:19.371382 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db205745-247e-4fdf-b65a-83f0f2520a90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:19.387099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:19.419464 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676c608-e281-437b-b3dd-4ec4cc8ff10f None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:19.441298 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db205745-247e-4fdf-b65a-83f0f2520a90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:19.442867 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db205745-247e-4fdf-b65a-83f0f2520a90 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:19.455241 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1f69e-4f62-4d67-a320-0e8c662867d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:19.456134 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1f69e-4f62-4d67-a320-0e8c662867d6 None admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:19.456413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1f69e-4f62-4d67-a320-0e8c662867d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:19.456685 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1f69e-4f62-4d67-a320-0e8c662867d6 None admin] PATH_INFO: `/v3/projects/6ceffdd955324aec84ba048d2137ae00/users/22a9592143014f3287cd75c60ca17237/roles/1705f116742c4217bb8f6eab32b40850` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:19.458919 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1f69e-4f62-4d67-a320-0e8c662867d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ceffdd955324aec84ba048d2137ae00, user_id=22a9592143014f3287cd75c60ca17237, role_id=1705f116742c4217bb8f6eab32b40850)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:19.460415 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1f69e-4f62-4d67-a320-0e8c662867d6 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:19.541202 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 538/1074] 10.222.0.24 () {58 vars in 1439 bytes} [Fri May 27 03:16:19 2022] PUT /identity/v3/projects/6ceffdd955324aec84ba048d2137ae00/users/22a9592143014f3287cd75c60ca17237/roles/1705f116742c4217bb8f6eab32b40850 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:16:19.545459 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 537/1075] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:16:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 03:16:19.546749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1f69e-4f62-4d67-a320-0e8c662867d6 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:19.650553 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:19.670586 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1f69e-4f62-4d67-a320-0e8c662867d6 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:19.708201 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d0a0a8-8eb6-451e-b7aa-7cfadfa24056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:19.709094 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0a0a8-8eb6-451e-b7aa-7cfadfa24056 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:19.709890 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0a0a8-8eb6-451e-b7aa-7cfadfa24056 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:19.710167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0a0a8-8eb6-451e-b7aa-7cfadfa24056 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:19.722084 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0a0a8-8eb6-451e-b7aa-7cfadfa24056 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:19.723624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d0a0a8-8eb6-451e-b7aa-7cfadfa24056 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:19.744731 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-85d0a0a8-8eb6-451e-b7aa-7cfadfa24056 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:16:19.744731 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:19.744731 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:19.744731 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:19.744731 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:19.744731 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:19.744731 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:19.744731 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:19.744731 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:19.744731 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:19.744731 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:19.744731 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:19.744731 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:16:19.744731 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:16:19.744731 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:16:19.745954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:16:19.745954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:19.745954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:19.745954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:19.745954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:19.745954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:19.745954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:19.745954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:19.745954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:16:19.745954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:19.745954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:19.745954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:19.745954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:19.745954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:19.745954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:19.745954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:19.745954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:19.747075 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:16:19.747075 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:16:19.747075 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:16:19.747075 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:16:19.748662 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 539/1076] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:16:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:16:19.754665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db205745-247e-4fdf-b65a-83f0f2520a90 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:19.801392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:19.855805 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db205745-247e-4fdf-b65a-83f0f2520a90 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:19.890886 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c2e80c-1c17-48c6-9a17-08037c9bc33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:19.891826 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2e80c-1c17-48c6-9a17-08037c9bc33d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:19.892256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2e80c-1c17-48c6-9a17-08037c9bc33d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:19.892675 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c2e80c-1c17-48c6-9a17-08037c9bc33d None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:19.893539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2e80c-1c17-48c6-9a17-08037c9bc33d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:19.893997 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2e80c-1c17-48c6-9a17-08037c9bc33d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:19.895646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c2e80c-1c17-48c6-9a17-08037c9bc33d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:19.908557 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 538/1077] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:21.284141 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8654e-6c94-4c25-9322-21ebb6262261 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:21.284726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8654e-6c94-4c25-9322-21ebb6262261 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:21.284726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8654e-6c94-4c25-9322-21ebb6262261 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:21.286028 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 540/1078] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:21.293350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9adee92-c880-4850-a951-6fc994086b2a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:21.293939 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9adee92-c880-4850-a951-6fc994086b2a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:21.293939 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9adee92-c880-4850-a951-6fc994086b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:21.314223 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-b9adee92-c880-4850-a951-6fc994086b2a None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:21.315624 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b9adee92-c880-4850-a951-6fc994086b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:21.398870 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 539/1079] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:16:21.420287 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a8654e-6c94-4c25-9322-21ebb6262261 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:21.518409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a8654e-6c94-4c25-9322-21ebb6262261 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:21.550101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771082a-606d-4307-a23f-e5256e16d28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:21.550903 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771082a-606d-4307-a23f-e5256e16d28d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:21.551181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771082a-606d-4307-a23f-e5256e16d28d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:21.551449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771082a-606d-4307-a23f-e5256e16d28d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:21.553732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:21.562068 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771082a-606d-4307-a23f-e5256e16d28d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:16:21.570240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-d771082a-606d-4307-a23f-e5256e16d28d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:16:21.570240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:21.570240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:21.570240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:21.570240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:21.570240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:21.570240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:21.570240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:21.570240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:21.570240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:21.570240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:21.570240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:21.570240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:16:21.570240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:16:21.570240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:16:21.570240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:16:21.571459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:16:21.571459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:16:21.571459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:16:21.571459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:16:21.571459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:21.571459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:21.571459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:21.571459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:21.571459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:21.571459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:21.571459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:21.571459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:16:21.571459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:21.571459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:21.571459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:21.571459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:21.577499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:21.577499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:21.577499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:21.577499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:21.577499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:16:21.577499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:16:21.577499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:16:21.577499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:16:21.577499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:16:21.577499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:16:21.577499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:16:21.577499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:16:21.577499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 541/1080] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:16:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:16:21.578244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9adee92-c880-4850-a951-6fc994086b2a None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:21.611463 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:21.690058 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9adee92-c880-4850-a951-6fc994086b2a None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:21.723184 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ff65c-670c-4344-88c2-86ff2739014c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:21.723842 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ff65c-670c-4344-88c2-86ff2739014c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:21.724122 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ff65c-670c-4344-88c2-86ff2739014c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:21.724396 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ff65c-670c-4344-88c2-86ff2739014c None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:21.725151 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10ff65c-670c-4344-88c2-86ff2739014c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:21.725456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10ff65c-670c-4344-88c2-86ff2739014c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:21.726818 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10ff65c-670c-4344-88c2-86ff2739014c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:21.734658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 540/1081] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:16:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:16:21.741037 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771082a-606d-4307-a23f-e5256e16d28d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:21.804102 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e93e4-cfa2-40bf-88a7-19bfa1ecc910 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:21.804389 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e93e4-cfa2-40bf-88a7-19bfa1ecc910 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:21.804668 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e93e4-cfa2-40bf-88a7-19bfa1ecc910 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:21.806276 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 541/1082] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:16:21.810904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82216c5a-8ac5-4161-9249-b4cbd4ab287a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:21.811201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82216c5a-8ac5-4161-9249-b4cbd4ab287a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:21.811489 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82216c5a-8ac5-4161-9249-b4cbd4ab287a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:21.828628 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-82216c5a-8ac5-4161-9249-b4cbd4ab287a None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:21.829920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-82216c5a-8ac5-4161-9249-b4cbd4ab287a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:21.840340 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771082a-606d-4307-a23f-e5256e16d28d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:21.873560 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5746af-ed4f-4b13-9400-d6519cec77a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:21.874430 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5746af-ed4f-4b13-9400-d6519cec77a9 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:21.874709 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5746af-ed4f-4b13-9400-d6519cec77a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:21.875029 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5746af-ed4f-4b13-9400-d6519cec77a9 None admin] PATH_INFO: `/v3/users/d9a554766bf649a2b5b3bffa155bc3da` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:21.876216 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5746af-ed4f-4b13-9400-d6519cec77a9 None admin] RBAC: Authorizing `identity:get_user(user_id=d9a554766bf649a2b5b3bffa155bc3da)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:21.877650 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5746af-ed4f-4b13-9400-d6519cec77a9 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:21.879307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 542/1083] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:16:21 2022] GET /identity/v3/users/d9a554766bf649a2b5b3bffa155bc3da => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:16:21.884835 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5746af-ed4f-4b13-9400-d6519cec77a9 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:21.895828 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 542/1084] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:16:21.919430 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82216c5a-8ac5-4161-9249-b4cbd4ab287a None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:21.949209 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:21.949361 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:21.984234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5746af-ed4f-4b13-9400-d6519cec77a9 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:22.014378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82216c5a-8ac5-4161-9249-b4cbd4ab287a None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:22.016787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b832994-7c97-436e-969b-1b8357787d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:22.017642 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b832994-7c97-436e-969b-1b8357787d43 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:22.017927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b832994-7c97-436e-969b-1b8357787d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:22.018205 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b832994-7c97-436e-969b-1b8357787d43 None admin] PATH_INFO: `/v3/projects/01eb74db1c404662890e270313372e72` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:22.019413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b832994-7c97-436e-969b-1b8357787d43 None admin] RBAC: Authorizing `identity:get_project(project_id=01eb74db1c404662890e270313372e72)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:22.020850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b832994-7c97-436e-969b-1b8357787d43 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:22.022864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 543/1085] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:16:21 2022] GET /identity/v3/projects/01eb74db1c404662890e270313372e72 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:22.028345 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b832994-7c97-436e-969b-1b8357787d43 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:22.054137 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46c8e2-c7bf-4754-9196-4c6a9a8193d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:22.054965 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46c8e2-c7bf-4754-9196-4c6a9a8193d4 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:22.055243 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46c8e2-c7bf-4754-9196-4c6a9a8193d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:22.055511 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46c8e2-c7bf-4754-9196-4c6a9a8193d4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:22.065309 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46c8e2-c7bf-4754-9196-4c6a9a8193d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:16:22.073442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-6f46c8e2-c7bf-4754-9196-4c6a9a8193d4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:16:22.073442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:22.073442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:22.073442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:22.073442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:22.073442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:22.073442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:22.073442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:22.073442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:22.073442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:22.073442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:22.073442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:22.073442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:16:22.073442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:16:22.073442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:16:22.073442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:16:22.074361 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:16:22.074361 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:16:22.074361 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:16:22.074361 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:16:22.074361 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:22.074361 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:22.074361 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:22.074361 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:22.074361 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:22.074361 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:22.074361 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:22.074361 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:16:22.074361 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:22.074361 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:22.074361 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:22.074361 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:22.075496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:22.075496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:22.075496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:22.075496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:22.075496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:16:22.075496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:16:22.075496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:16:22.075496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:16:22.075496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:16:22.075496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:16:22.075496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:16:22.075496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:22.075496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 543/1086] 10.222.0.24 () {56 vars in 1120 bytes} [Fri May 27 03:16:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:16:22.080900 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46c8e2-c7bf-4754-9196-4c6a9a8193d4 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:22.131313 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b832994-7c97-436e-969b-1b8357787d43 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:22.158633 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:22.164797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a4c75-6c39-4fca-bbce-8397a85b8689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:22.165683 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a4c75-6c39-4fca-bbce-8397a85b8689 None admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:22.165961 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a4c75-6c39-4fca-bbce-8397a85b8689 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:22.166228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a4c75-6c39-4fca-bbce-8397a85b8689 None admin] PATH_INFO: `/v3/projects/01eb74db1c404662890e270313372e72/users/d9a554766bf649a2b5b3bffa155bc3da/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:22.168193 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3a4c75-6c39-4fca-bbce-8397a85b8689 None admin] RBAC: Authorizing `identity:create_grant(project_id=01eb74db1c404662890e270313372e72, user_id=d9a554766bf649a2b5b3bffa155bc3da, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:22.168754 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:22.169628 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3a4c75-6c39-4fca-bbce-8397a85b8689 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:22.180795 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46c8e2-c7bf-4754-9196-4c6a9a8193d4 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:22.213862 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991e5c5-fdb0-42ec-af98-34c6d8e299fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:22.214462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991e5c5-fdb0-42ec-af98-34c6d8e299fd None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:22.214746 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991e5c5-fdb0-42ec-af98-34c6d8e299fd None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:22.215050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991e5c5-fdb0-42ec-af98-34c6d8e299fd None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:22.215791 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991e5c5-fdb0-42ec-af98-34c6d8e299fd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:22.216099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991e5c5-fdb0-42ec-af98-34c6d8e299fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:22.217442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991e5c5-fdb0-42ec-af98-34c6d8e299fd None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:22.225897 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 544/1087] 10.222.0.24 () {56 vars in 1115 bytes} [Fri May 27 03:16:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:16:22.231631 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991e5c5-fdb0-42ec-af98-34c6d8e299fd None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:22.303991 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 544/1088] 10.222.0.24 () {58 vars in 1439 bytes} [Fri May 27 03:16:22 2022] PUT /identity/v3/projects/01eb74db1c404662890e270313372e72/users/d9a554766bf649a2b5b3bffa155bc3da/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:16:22.327358 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991e5c5-fdb0-42ec-af98-34c6d8e299fd None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:22.364067 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab2561-342e-4422-9663-b1599283a0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:22.364888 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab2561-342e-4422-9663-b1599283a0a8 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:22.365174 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab2561-342e-4422-9663-b1599283a0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:22.365453 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab2561-342e-4422-9663-b1599283a0a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:22.377442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab2561-342e-4422-9663-b1599283a0a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:22.378963 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceab2561-342e-4422-9663-b1599283a0a8 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:22.397728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-ceab2561-342e-4422-9663-b1599283a0a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:16:22.397728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:22.397728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:22.397728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:22.397728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:22.397728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:22.397728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:22.397728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:22.397728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:22.397728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:22.397728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:22.397728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:22.397728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:16:22.397728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:16:22.397728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:16:22.398752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:16:22.398752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:22.398752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:22.398752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:22.398752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:22.398752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:22.398752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:22.398752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:22.398752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:16:22.398752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:22.398752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:22.398752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:22.398752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:22.398752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:22.398752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:22.398752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:22.398752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:22.399714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:16:22.399714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:16:22.399714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:16:22.399714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:22.399714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 545/1089] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:16:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:16:22.404953 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a4c75-6c39-4fca-bbce-8397a85b8689 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:22.508233 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a4c75-6c39-4fca-bbce-8397a85b8689 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:22.542524 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd38505-c188-4729-a58d-ee26605f2aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:22.543215 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd38505-c188-4729-a58d-ee26605f2aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:22.543498 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd38505-c188-4729-a58d-ee26605f2aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:22.543771 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd38505-c188-4729-a58d-ee26605f2aa0 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:22.544484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd38505-c188-4729-a58d-ee26605f2aa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:22.544789 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd38505-c188-4729-a58d-ee26605f2aa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:22.546162 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd38505-c188-4729-a58d-ee26605f2aa0 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:22.557607 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 545/1090] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:22.563576 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab2561-342e-4422-9663-b1599283a0a8 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:22.566533 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:22.662947 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab2561-342e-4422-9663-b1599283a0a8 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:22.701872 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e843d9-242b-4928-a78d-cb8b768e481e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:22.703303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e843d9-242b-4928-a78d-cb8b768e481e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:22.703583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e843d9-242b-4928-a78d-cb8b768e481e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:22.703853 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e843d9-242b-4928-a78d-cb8b768e481e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:22.712894 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e843d9-242b-4928-a78d-cb8b768e481e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:22.714325 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e843d9-242b-4928-a78d-cb8b768e481e None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:22.728156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-93e843d9-242b-4928-a78d-cb8b768e481e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:16:22.728156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:22.728156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:22.728156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:22.728156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:22.728156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:22.728156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:22.728156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:22.728156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:22.728156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:22.728156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:22.728156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:22.728156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:16:22.728156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:16:22.728156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:16:22.728156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:16:22.729147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:22.729147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:22.729147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:16:22.729147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:16:22.729147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:16:22.729147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:16:22.729147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:22.729147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:22.729147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:22.729147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:22.729147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:22.729147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:16:22.729147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:22.729147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:22.729147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:22.729147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:22.729147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:22.729147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:22.730573 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:22.730573 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:22.730573 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:16:22.730573 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:16:22.730573 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:16:22.730573 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:16:22.730573 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:16:22.730573 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:16:22.730573 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:16:22.730573 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:22.730573 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 546/1091] 10.222.0.24 () {56 vars in 1093 bytes} [Fri May 27 03:16:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:16:22.735058 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd38505-c188-4729-a58d-ee26605f2aa0 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:22.787361 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:22.840763 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd38505-c188-4729-a58d-ee26605f2aa0 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:22.874157 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9db79b-d931-402c-9507-c262471bb17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:22.874817 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9db79b-d931-402c-9507-c262471bb17c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:22.875133 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9db79b-d931-402c-9507-c262471bb17c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:22.875399 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9db79b-d931-402c-9507-c262471bb17c None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:22.876217 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9db79b-d931-402c-9507-c262471bb17c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:22.876524 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9db79b-d931-402c-9507-c262471bb17c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:22.877937 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9db79b-d931-402c-9507-c262471bb17c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:22.894959 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 546/1092] 10.222.0.24 () {56 vars in 1133 bytes} [Fri May 27 03:16:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:22.901033 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e843d9-242b-4928-a78d-cb8b768e481e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:22.967702 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:23.006453 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e843d9-242b-4928-a78d-cb8b768e481e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:23.038648 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf0f05-d20b-4bce-9924-cfdac6de3c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:23.039235 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf0f05-d20b-4bce-9924-cfdac6de3c64 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:23.039514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf0f05-d20b-4bce-9924-cfdac6de3c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:23.039785 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf0f05-d20b-4bce-9924-cfdac6de3c64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:23.106371 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf0f05-d20b-4bce-9924-cfdac6de3c64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:23.107856 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf0f05-d20b-4bce-9924-cfdac6de3c64 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:23.164942 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:23.203499 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 547/1093] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:16:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:16:23.209298 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9db79b-d931-402c-9507-c262471bb17c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:23.304677 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9db79b-d931-402c-9507-c262471bb17c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:23.313449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:23.338479 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06707f32-222a-4084-9f78-970507ee2531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:23.339302 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06707f32-222a-4084-9f78-970507ee2531 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:23.339582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06707f32-222a-4084-9f78-970507ee2531 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:23.339856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06707f32-222a-4084-9f78-970507ee2531 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:23.350388 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06707f32-222a-4084-9f78-970507ee2531 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:23.351864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06707f32-222a-4084-9f78-970507ee2531 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:23.369518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-06707f32-222a-4084-9f78-970507ee2531 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:16:23.369518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:23.369518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:23.369518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:23.369518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:23.369518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:23.369518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:23.369518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:23.369518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:23.369518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:23.369518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:23.369518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:23.369518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:16:23.369518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:16:23.369518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:16:23.370432 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:16:23.370432 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:23.370432 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:23.370432 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:23.370432 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:23.370432 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:23.370432 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:23.370432 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:23.370432 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:16:23.370432 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:23.370432 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:23.370432 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:23.370432 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:23.370432 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:23.370432 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:23.370432 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:23.370432 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:23.371476 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:16:23.371476 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:16:23.371476 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:16:23.371476 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:16:23.371476 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 547/1094] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:16:23.376356 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf0f05-d20b-4bce-9924-cfdac6de3c64 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:23.475379 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf0f05-d20b-4bce-9924-cfdac6de3c64 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:23.508701 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f414fee-a9af-480f-b7bb-c2d9919bdafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:23.509363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f414fee-a9af-480f-b7bb-c2d9919bdafb None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:23.509646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f414fee-a9af-480f-b7bb-c2d9919bdafb None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:23.509922 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f414fee-a9af-480f-b7bb-c2d9919bdafb None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:23.510639 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f414fee-a9af-480f-b7bb-c2d9919bdafb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:23.510984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f414fee-a9af-480f-b7bb-c2d9919bdafb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:23.512378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f414fee-a9af-480f-b7bb-c2d9919bdafb None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:23.524113 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 548/1095] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:23.529660 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06707f32-222a-4084-9f78-970507ee2531 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:23.624981 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06707f32-222a-4084-9f78-970507ee2531 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:23.657712 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c9d73-69ae-4118-a442-c5d60c7c284b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:23.658492 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c9d73-69ae-4118-a442-c5d60c7c284b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:23.658770 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c9d73-69ae-4118-a442-c5d60c7c284b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:23.659069 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c9d73-69ae-4118-a442-c5d60c7c284b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:23.665474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c9d73-69ae-4118-a442-c5d60c7c284b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:23.666941 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c9d73-69ae-4118-a442-c5d60c7c284b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:23.680278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-0e9c9d73-69ae-4118-a442-c5d60c7c284b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:16:23.680278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:23.680278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:23.680278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:23.680278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:23.680278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:23.680278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:23.680278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:23.680278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:23.680278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:23.680278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:23.680278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:23.680278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:16:23.680278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:16:23.680278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:16:23.681239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:16:23.681239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:23.681239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:23.681239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:23.681239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:23.681239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:23.681239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:23.681239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:23.681239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:16:23.681239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:23.681239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:23.681239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:23.681239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:23.681239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:23.681239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:23.681239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:23.681239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:23.682146 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:16:23.682146 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:16:23.682146 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:16:23.682146 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:16:23.682146 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:16:23.682146 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:16:23.682146 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:16:23.682146 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:16:23.683050 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 548/1096] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:16:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:16:23.687199 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f414fee-a9af-480f-b7bb-c2d9919bdafb None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:23.712327 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:23.784709 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f414fee-a9af-480f-b7bb-c2d9919bdafb None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:23.816874 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b112cae-99c9-48c1-9f3e-938d267e2d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:23.817526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b112cae-99c9-48c1-9f3e-938d267e2d34 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:23.817808 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b112cae-99c9-48c1-9f3e-938d267e2d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:23.818083 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b112cae-99c9-48c1-9f3e-938d267e2d34 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:23.819454 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b112cae-99c9-48c1-9f3e-938d267e2d34 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:23.819760 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b112cae-99c9-48c1-9f3e-938d267e2d34 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:23.821172 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b112cae-99c9-48c1-9f3e-938d267e2d34 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:23.833060 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 549/1097] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:16:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:23.839121 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c9d73-69ae-4118-a442-c5d60c7c284b None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:23.860990 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:23.937472 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c9d73-69ae-4118-a442-c5d60c7c284b None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:23.971183 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01d8fb-88b5-459f-bf12-e5a9fd16b861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:23.971838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01d8fb-88b5-459f-bf12-e5a9fd16b861 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:23.972119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01d8fb-88b5-459f-bf12-e5a9fd16b861 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:23.972395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01d8fb-88b5-459f-bf12-e5a9fd16b861 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:23.973223 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b01d8fb-88b5-459f-bf12-e5a9fd16b861 None admin] RBAC: Adding query filter params (role.id=1705f116742c4217bb8f6eab32b40850, scope.project.id=6ceffdd955324aec84ba048d2137ae00, user.id=22a9592143014f3287cd75c60ca17237) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:23.973544 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b01d8fb-88b5-459f-bf12-e5a9fd16b861 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:23.975032 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b01d8fb-88b5-459f-bf12-e5a9fd16b861 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:23.980829 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 549/1098] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:16:23 2022] GET /identity/v3/role_assignments?user.id=22a9592143014f3287cd75c60ca17237&scope.project.id=6ceffdd955324aec84ba048d2137ae00&role.id=1705f116742c4217bb8f6eab32b40850 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:24.504829 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85929f-1f54-463f-ab9b-b4fafad4d494 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:24.505343 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85929f-1f54-463f-ab9b-b4fafad4d494 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:24.505463 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85929f-1f54-463f-ab9b-b4fafad4d494 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:24.506809 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 550/1099] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:24.512340 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e96c1-2633-42d1-9fb0-e93e0cdbfb81 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:24.512744 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e96c1-2633-42d1-9fb0-e93e0cdbfb81 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:24.512882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e96c1-2633-42d1-9fb0-e93e0cdbfb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:24.529874 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-287e96c1-2633-42d1-9fb0-e93e0cdbfb81 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:24.531074 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-287e96c1-2633-42d1-9fb0-e93e0cdbfb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:24.601997 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 550/1100] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:16:24.624801 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad85929f-1f54-463f-ab9b-b4fafad4d494 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:24.726968 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:24.747258 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad85929f-1f54-463f-ab9b-b4fafad4d494 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:24.786936 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90937578-5bf0-481f-9488-6bcdb3b2fcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:24.787739 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90937578-5bf0-481f-9488-6bcdb3b2fcaf None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:24.788018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90937578-5bf0-481f-9488-6bcdb3b2fcaf None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:24.788290 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90937578-5bf0-481f-9488-6bcdb3b2fcaf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:24.803063 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90937578-5bf0-481f-9488-6bcdb3b2fcaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:16:24.810519 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-90937578-5bf0-481f-9488-6bcdb3b2fcaf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:16:24.810519 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:24.810519 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:24.810519 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:24.810519 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:24.810519 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:24.810519 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:24.810519 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:24.810519 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:24.810519 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:24.810519 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:24.810519 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:24.810519 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:16:24.810519 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:16:24.810519 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:16:24.810519 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:16:24.811488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:16:24.811488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:16:24.811488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:16:24.811488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:16:24.811488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:24.811488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:24.811488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:24.811488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:24.811488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:24.811488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:24.811488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:24.811488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:16:24.811488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:24.811488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:24.811488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:24.811488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:24.812330 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:24.812330 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:24.812330 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:24.812330 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:24.812330 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:16:24.812330 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:16:24.812330 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:16:24.812330 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:16:24.812330 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:16:24.812330 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:16:24.812330 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:16:24.812330 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:24.812330 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 551/1101] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:16:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:16:24.818148 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287e96c1-2633-42d1-9fb0-e93e0cdbfb81 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:24.835548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:24.921979 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287e96c1-2633-42d1-9fb0-e93e0cdbfb81 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:24.956028 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbbe94c-b652-4078-8007-d167f694bd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:24.956687 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbe94c-b652-4078-8007-d167f694bd7e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:24.956965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbe94c-b652-4078-8007-d167f694bd7e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:24.957234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbe94c-b652-4078-8007-d167f694bd7e None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:24.957970 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbbe94c-b652-4078-8007-d167f694bd7e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:24.958274 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbbe94c-b652-4078-8007-d167f694bd7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:24.959690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbbe94c-b652-4078-8007-d167f694bd7e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:24.967588 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 551/1102] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:16:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:24.973528 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90937578-5bf0-481f-9488-6bcdb3b2fcaf None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:25.072984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90937578-5bf0-481f-9488-6bcdb3b2fcaf None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:25.105628 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe2ec3-0a84-4359-b87b-e74db8d9aee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:25.106486 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe2ec3-0a84-4359-b87b-e74db8d9aee9 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:25.106761 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe2ec3-0a84-4359-b87b-e74db8d9aee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:25.107054 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe2ec3-0a84-4359-b87b-e74db8d9aee9 None admin] PATH_INFO: `/v3/users/d9a554766bf649a2b5b3bffa155bc3da` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:25.108217 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe2ec3-0a84-4359-b87b-e74db8d9aee9 None admin] RBAC: Authorizing `identity:get_user(user_id=d9a554766bf649a2b5b3bffa155bc3da)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:25.109632 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fe2ec3-0a84-4359-b87b-e74db8d9aee9 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:25.111270 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 552/1103] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:16:24 2022] GET /identity/v3/users/d9a554766bf649a2b5b3bffa155bc3da => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:25.117675 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbbe94c-b652-4078-8007-d167f694bd7e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:25.191127 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:25.213686 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbbe94c-b652-4078-8007-d167f694bd7e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:25.245322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11701ed3-e624-411b-b193-fee64c72432c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:25.245948 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11701ed3-e624-411b-b193-fee64c72432c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:25.246226 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11701ed3-e624-411b-b193-fee64c72432c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:25.246496 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11701ed3-e624-411b-b193-fee64c72432c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:25.313234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11701ed3-e624-411b-b193-fee64c72432c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:25.314674 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11701ed3-e624-411b-b193-fee64c72432c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:25.387803 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:25.409581 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 552/1104] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:16:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:16:25.415718 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe2ec3-0a84-4359-b87b-e74db8d9aee9 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:25.429258 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:25.513917 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe2ec3-0a84-4359-b87b-e74db8d9aee9 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:25.547019 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2187d55-3f50-4f05-9e40-7458a3c4bd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:25.547863 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2187d55-3f50-4f05-9e40-7458a3c4bd39 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:25.548137 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2187d55-3f50-4f05-9e40-7458a3c4bd39 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:25.548407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2187d55-3f50-4f05-9e40-7458a3c4bd39 None admin] PATH_INFO: `/v3/projects/01eb74db1c404662890e270313372e72` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:25.549566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2187d55-3f50-4f05-9e40-7458a3c4bd39 None admin] RBAC: Authorizing `identity:get_project(project_id=01eb74db1c404662890e270313372e72)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:25.551006 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2187d55-3f50-4f05-9e40-7458a3c4bd39 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:25.553159 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 553/1105] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:16:25 2022] GET /identity/v3/projects/01eb74db1c404662890e270313372e72 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:25.559203 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11701ed3-e624-411b-b193-fee64c72432c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:25.664602 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11701ed3-e624-411b-b193-fee64c72432c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:25.697351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184aba7-2e8e-4587-a78e-470e101e3e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:25.697996 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184aba7-2e8e-4587-a78e-470e101e3e90 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:25.698276 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184aba7-2e8e-4587-a78e-470e101e3e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:25.698549 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184aba7-2e8e-4587-a78e-470e101e3e90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:25.699397 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9184aba7-2e8e-4587-a78e-470e101e3e90 None admin] RBAC: Adding query filter params (role.id=e5a908bd3ed5445b82a6053246533b88, scope.project.id=01eb74db1c404662890e270313372e72, user.id=d9a554766bf649a2b5b3bffa155bc3da) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:25.699962 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9184aba7-2e8e-4587-a78e-470e101e3e90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:25.701377 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9184aba7-2e8e-4587-a78e-470e101e3e90 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:25.706989 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 553/1106] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:16:25 2022] GET /identity/v3/role_assignments?user.id=d9a554766bf649a2b5b3bffa155bc3da&scope.project.id=01eb74db1c404662890e270313372e72&role.id=e5a908bd3ed5445b82a6053246533b88 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:27.897238 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14c1d5-72c4-41a8-b9d2-c3364ece599f None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:27.897767 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14c1d5-72c4-41a8-b9d2-c3364ece599f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:27.897767 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14c1d5-72c4-41a8-b9d2-c3364ece599f None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:27.899151 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 554/1107] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:27.904260 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817eecf9-a78e-4a8e-bb43-b7cb2d603664 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:27.904634 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817eecf9-a78e-4a8e-bb43-b7cb2d603664 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:27.904822 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817eecf9-a78e-4a8e-bb43-b7cb2d603664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:27.921633 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-817eecf9-a78e-4a8e-bb43-b7cb2d603664 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:27.922908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-817eecf9-a78e-4a8e-bb43-b7cb2d603664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:27.956166 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:27.987222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 554/1108] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:16:28.011644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14c1d5-72c4-41a8-b9d2-c3364ece599f None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:28.088051 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:28.109397 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14c1d5-72c4-41a8-b9d2-c3364ece599f None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:28.141828 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355ad6e-9af4-4cdb-9388-90d9b00eeebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:28.142599 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355ad6e-9af4-4cdb-9388-90d9b00eeebe None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:28.142901 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355ad6e-9af4-4cdb-9388-90d9b00eeebe None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:28.143174 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355ad6e-9af4-4cdb-9388-90d9b00eeebe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:28.144319 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355ad6e-9af4-4cdb-9388-90d9b00eeebe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:28.145729 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355ad6e-9af4-4cdb-9388-90d9b00eeebe None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:28.147436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 555/1109] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:16:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:28.153051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817eecf9-a78e-4a8e-bb43-b7cb2d603664 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:28.248289 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817eecf9-a78e-4a8e-bb43-b7cb2d603664 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:28.280519 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5d946-5df4-4ac4-9243-c1b6c238910c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:28.281177 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5d946-5df4-4ac4-9243-c1b6c238910c None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:28.281458 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5d946-5df4-4ac4-9243-c1b6c238910c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:28.281730 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5d946-5df4-4ac4-9243-c1b6c238910c None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:28.282444 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5d946-5df4-4ac4-9243-c1b6c238910c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:28.283895 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5d946-5df4-4ac4-9243-c1b6c238910c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:28.408423 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 555/1110] 10.222.0.24 () {60 vars in 1132 bytes} [Fri May 27 03:16:28 2022] POST /identity/v3/users => generated 286 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:16:30.621603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f07e611-964e-47e1-9008-a65957065588 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:30.622123 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f07e611-964e-47e1-9008-a65957065588 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:30.622123 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f07e611-964e-47e1-9008-a65957065588 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:30.623528 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 556/1111] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:30.628630 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef099d5-a239-48a8-868e-2650d82c8fc4 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:30.628997 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef099d5-a239-48a8-868e-2650d82c8fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:30.629180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef099d5-a239-48a8-868e-2650d82c8fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:30.645981 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-3ef099d5-a239-48a8-868e-2650d82c8fc4 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:30.647191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3ef099d5-a239-48a8-868e-2650d82c8fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:30.666052 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:30.710675 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 556/1112] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:16:30.733432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f07e611-964e-47e1-9008-a65957065588 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:30.830991 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f07e611-964e-47e1-9008-a65957065588 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:30.864272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30707b96-bffd-4280-baf9-7e84dcf91184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:30.865202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30707b96-bffd-4280-baf9-7e84dcf91184 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:30.865479 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30707b96-bffd-4280-baf9-7e84dcf91184 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:30.865749 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30707b96-bffd-4280-baf9-7e84dcf91184 None admin] PATH_INFO: `/v3/roles/d7133ce917f0431883775551ab086d5d` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:30.867227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30707b96-bffd-4280-baf9-7e84dcf91184 None admin] RBAC: Authorizing `identity:get_role(role_id=d7133ce917f0431883775551ab086d5d)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:30.868576 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30707b96-bffd-4280-baf9-7e84dcf91184 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:30.869896 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 557/1113] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:16:30 2022] GET /identity/v3/roles/d7133ce917f0431883775551ab086d5d => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:30.875677 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef099d5-a239-48a8-868e-2650d82c8fc4 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:30.968912 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef099d5-a239-48a8-868e-2650d82c8fc4 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:31.000736 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b27f6-170b-4b4e-a5e7-3fcb594b03fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:31.001590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b27f6-170b-4b4e-a5e7-3fcb594b03fa None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:31.001868 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b27f6-170b-4b4e-a5e7-3fcb594b03fa None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:31.002137 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6b27f6-170b-4b4e-a5e7-3fcb594b03fa None admin] PATH_INFO: `/v3/users/be785129233e40e9875390e5de7d77d5` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:31.015148 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b27f6-170b-4b4e-a5e7-3fcb594b03fa None admin] RBAC: Authorizing `identity:get_user(user_id=be785129233e40e9875390e5de7d77d5)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:31.016627 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6b27f6-170b-4b4e-a5e7-3fcb594b03fa None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:31.018340 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 557/1114] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:16:30 2022] GET /identity/v3/users/be785129233e40e9875390e5de7d77d5 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:31.025350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30707b96-bffd-4280-baf9-7e84dcf91184 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:31.044073 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:31.132020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30707b96-bffd-4280-baf9-7e84dcf91184 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:31.164810 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd12c539-9119-4e1d-adf2-b6b322783680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:31.165445 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12c539-9119-4e1d-adf2-b6b322783680 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:31.165727 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12c539-9119-4e1d-adf2-b6b322783680 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:31.166003 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12c539-9119-4e1d-adf2-b6b322783680 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:31.231275 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd12c539-9119-4e1d-adf2-b6b322783680 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:31.232706 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd12c539-9119-4e1d-adf2-b6b322783680 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:31.246911 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:31.328113 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 558/1115] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:16:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:16:31.334039 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b27f6-170b-4b4e-a5e7-3fcb594b03fa None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:31.348901 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:31.441971 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6b27f6-170b-4b4e-a5e7-3fcb594b03fa None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:31.474126 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8528c9-6691-4a15-8406-d227fa29708b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:31.474997 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8528c9-6691-4a15-8406-d227fa29708b None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:31.475274 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8528c9-6691-4a15-8406-d227fa29708b None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:31.475545 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8528c9-6691-4a15-8406-d227fa29708b None admin] PATH_INFO: `/v3/projects/01eb74db1c404662890e270313372e72` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:31.476701 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8528c9-6691-4a15-8406-d227fa29708b None admin] RBAC: Authorizing `identity:get_project(project_id=01eb74db1c404662890e270313372e72)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:31.478124 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8528c9-6691-4a15-8406-d227fa29708b None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:31.479816 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 558/1116] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:16:31 2022] GET /identity/v3/projects/01eb74db1c404662890e270313372e72 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:31.485844 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd12c539-9119-4e1d-adf2-b6b322783680 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:31.581226 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd12c539-9119-4e1d-adf2-b6b322783680 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:31.594101 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:31.613248 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d7403-f1a9-40c3-9e55-55787bd02427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:31.613879 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d7403-f1a9-40c3-9e55-55787bd02427 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:31.614157 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d7403-f1a9-40c3-9e55-55787bd02427 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:31.614426 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d7403-f1a9-40c3-9e55-55787bd02427 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:31.615308 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d7403-f1a9-40c3-9e55-55787bd02427 None admin] RBAC: Adding query filter params (role.id=d7133ce917f0431883775551ab086d5d, scope.project.id=01eb74db1c404662890e270313372e72, user.id=be785129233e40e9875390e5de7d77d5) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:31.615615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d7403-f1a9-40c3-9e55-55787bd02427 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:31.617038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d7403-f1a9-40c3-9e55-55787bd02427 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:31.622146 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 559/1117] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:16:31 2022] GET /identity/v3/role_assignments?user.id=be785129233e40e9875390e5de7d77d5&scope.project.id=01eb74db1c404662890e270313372e72&role.id=d7133ce917f0431883775551ab086d5d => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:33.787050 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9e5fe-549d-40ea-8b35-788fab66fb11 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:33.787616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9e5fe-549d-40ea-8b35-788fab66fb11 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:33.787616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9e5fe-549d-40ea-8b35-788fab66fb11 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:33.788947 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 559/1118] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:33.794185 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056923a-235d-48b9-a73f-70e4e626e6a7 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:33.794553 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056923a-235d-48b9-a73f-70e4e626e6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:33.794743 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056923a-235d-48b9-a73f-70e4e626e6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:33.811696 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-f056923a-235d-48b9-a73f-70e4e626e6a7 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:33.812879 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f056923a-235d-48b9-a73f-70e4e626e6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:33.877820 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 560/1119] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:16:33.899816 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9e5fe-549d-40ea-8b35-788fab66fb11 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:33.978716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:33.996699 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9e5fe-549d-40ea-8b35-788fab66fb11 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:34.028253 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e0450-3fba-4c7a-88b8-b0dc2717ab4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:34.029106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e0450-3fba-4c7a-88b8-b0dc2717ab4d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:34.029387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e0450-3fba-4c7a-88b8-b0dc2717ab4d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:34.029662 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e0450-3fba-4c7a-88b8-b0dc2717ab4d None admin] PATH_INFO: `/v3/roles/d7133ce917f0431883775551ab086d5d` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:34.031152 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757e0450-3fba-4c7a-88b8-b0dc2717ab4d None admin] RBAC: Authorizing `identity:get_role(role_id=d7133ce917f0431883775551ab086d5d)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:34.032506 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757e0450-3fba-4c7a-88b8-b0dc2717ab4d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:34.033846 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 560/1120] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:16:33 2022] GET /identity/v3/roles/d7133ce917f0431883775551ab086d5d => generated 223 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:16:34.040282 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f056923a-235d-48b9-a73f-70e4e626e6a7 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:34.135599 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f056923a-235d-48b9-a73f-70e4e626e6a7 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:34.144407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:34.172267 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5fa1c-e584-46e2-9753-836456437681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:34.173122 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5fa1c-e584-46e2-9753-836456437681 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:34.173400 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5fa1c-e584-46e2-9753-836456437681 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:34.173666 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5fa1c-e584-46e2-9753-836456437681 None admin] PATH_INFO: `/v3/users/be785129233e40e9875390e5de7d77d5` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:34.174824 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a5fa1c-e584-46e2-9753-836456437681 None admin] RBAC: Authorizing `identity:get_user(user_id=be785129233e40e9875390e5de7d77d5)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:34.176274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a5fa1c-e584-46e2-9753-836456437681 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:34.178451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 561/1121] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:16:34 2022] GET /identity/v3/users/be785129233e40e9875390e5de7d77d5 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:34.184151 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e0450-3fba-4c7a-88b8-b0dc2717ab4d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:34.278317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e0450-3fba-4c7a-88b8-b0dc2717ab4d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:34.310867 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b28b80-18a7-4717-aced-427fc641dfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:34.311657 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b28b80-18a7-4717-aced-427fc641dfeb None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:34.311932 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b28b80-18a7-4717-aced-427fc641dfeb None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:34.312200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b28b80-18a7-4717-aced-427fc641dfeb None admin] PATH_INFO: `/v3/projects/01eb74db1c404662890e270313372e72` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:34.313347 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b28b80-18a7-4717-aced-427fc641dfeb None admin] RBAC: Authorizing `identity:get_project(project_id=01eb74db1c404662890e270313372e72)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:34.314756 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b28b80-18a7-4717-aced-427fc641dfeb None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:34.316418 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 561/1122] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:16:34 2022] GET /identity/v3/projects/01eb74db1c404662890e270313372e72 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:34.322179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5fa1c-e584-46e2-9753-836456437681 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:34.426531 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5fa1c-e584-46e2-9753-836456437681 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:34.459624 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d0abf-e499-441b-9759-5c56c665ad68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:34.460505 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d0abf-e499-441b-9759-5c56c665ad68 None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:34.460784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d0abf-e499-441b-9759-5c56c665ad68 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:34.461057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d0abf-e499-441b-9759-5c56c665ad68 None admin] PATH_INFO: `/v3/projects/01eb74db1c404662890e270313372e72/users/be785129233e40e9875390e5de7d77d5/roles/d7133ce917f0431883775551ab086d5d` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:34.463261 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d0abf-e499-441b-9759-5c56c665ad68 None admin] RBAC: Authorizing `identity:create_grant(project_id=01eb74db1c404662890e270313372e72, user_id=be785129233e40e9875390e5de7d77d5, role_id=d7133ce917f0431883775551ab086d5d)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:34.464687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d0abf-e499-441b-9759-5c56c665ad68 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:34.530787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 562/1123] 10.222.0.24 () {58 vars in 1439 bytes} [Fri May 27 03:16:34 2022] PUT /identity/v3/projects/01eb74db1c404662890e270313372e72/users/be785129233e40e9875390e5de7d77d5/roles/d7133ce917f0431883775551ab086d5d => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:16:36.675342 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ee486-f5d2-4a97-a45d-827386c1919c None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:36.675895 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ee486-f5d2-4a97-a45d-827386c1919c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:36.675895 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ee486-f5d2-4a97-a45d-827386c1919c None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:36.677745 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 562/1124] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:36.682857 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7133e5d-73e9-4bce-9379-75bb8aba2573 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:36.683221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7133e5d-73e9-4bce-9379-75bb8aba2573 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:36.683435 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7133e5d-73e9-4bce-9379-75bb8aba2573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:36.700605 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-a7133e5d-73e9-4bce-9379-75bb8aba2573 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:36.701780 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a7133e5d-73e9-4bce-9379-75bb8aba2573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:36.729240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:36.765295 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 563/1125] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:16:36.790716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ee486-f5d2-4a97-a45d-827386c1919c None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:36.812989 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:36.894707 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ee486-f5d2-4a97-a45d-827386c1919c None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:36.927811 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35732769-e7f4-45ba-afee-3ab1bece09d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:36.928600 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35732769-e7f4-45ba-afee-3ab1bece09d9 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:36.928877 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35732769-e7f4-45ba-afee-3ab1bece09d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:36.929147 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35732769-e7f4-45ba-afee-3ab1bece09d9 None admin] PATH_INFO: `/v3/roles/d7133ce917f0431883775551ab086d5d` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:36.930606 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35732769-e7f4-45ba-afee-3ab1bece09d9 None admin] RBAC: Authorizing `identity:get_role(role_id=d7133ce917f0431883775551ab086d5d)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:36.932009 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35732769-e7f4-45ba-afee-3ab1bece09d9 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:36.933366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 563/1126] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:16:36 2022] GET /identity/v3/roles/d7133ce917f0431883775551ab086d5d => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:36.939270 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7133e5d-73e9-4bce-9379-75bb8aba2573 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:37.033689 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7133e5d-73e9-4bce-9379-75bb8aba2573 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:37.065275 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c48cd-7f20-40a0-b693-8f0fd8736ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:37.066117 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c48cd-7f20-40a0-b693-8f0fd8736ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:37.066394 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c48cd-7f20-40a0-b693-8f0fd8736ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:37.066665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c48cd-7f20-40a0-b693-8f0fd8736ba5 None admin] PATH_INFO: `/v3/users/be785129233e40e9875390e5de7d77d5` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:37.068378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c48cd-7f20-40a0-b693-8f0fd8736ba5 None admin] RBAC: Authorizing `identity:get_user(user_id=be785129233e40e9875390e5de7d77d5)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:37.069783 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c48cd-7f20-40a0-b693-8f0fd8736ba5 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:37.071390 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 564/1127] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:16:36 2022] GET /identity/v3/users/be785129233e40e9875390e5de7d77d5 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:37.077292 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35732769-e7f4-45ba-afee-3ab1bece09d9 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:37.163121 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:37.177587 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35732769-e7f4-45ba-afee-3ab1bece09d9 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:37.211436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a561752-e905-4f26-8752-8c6cc3080c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:37.212009 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a561752-e905-4f26-8752-8c6cc3080c18 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:37.212288 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a561752-e905-4f26-8752-8c6cc3080c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:37.212562 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a561752-e905-4f26-8752-8c6cc3080c18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:37.276002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a561752-e905-4f26-8752-8c6cc3080c18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:37.277440 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a561752-e905-4f26-8752-8c6cc3080c18 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:37.356205 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:37.370730 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 564/1128] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:16:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:16:37.376665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c48cd-7f20-40a0-b693-8f0fd8736ba5 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:37.415445 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:37.473657 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c48cd-7f20-40a0-b693-8f0fd8736ba5 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:37.506566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221546a6-82aa-4be6-8309-62acaa0cecc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:37.507456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221546a6-82aa-4be6-8309-62acaa0cecc9 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:37.507737 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221546a6-82aa-4be6-8309-62acaa0cecc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:37.508009 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221546a6-82aa-4be6-8309-62acaa0cecc9 None admin] PATH_INFO: `/v3/projects/01eb74db1c404662890e270313372e72` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:37.509181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221546a6-82aa-4be6-8309-62acaa0cecc9 None admin] RBAC: Authorizing `identity:get_project(project_id=01eb74db1c404662890e270313372e72)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:37.510601 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221546a6-82aa-4be6-8309-62acaa0cecc9 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:37.512322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 565/1129] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:16:37 2022] GET /identity/v3/projects/01eb74db1c404662890e270313372e72 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:37.518484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a561752-e905-4f26-8752-8c6cc3080c18 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:37.613440 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a561752-e905-4f26-8752-8c6cc3080c18 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:37.646124 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3acc5c-fc07-4702-9191-bf3766f9baca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:37.646766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3acc5c-fc07-4702-9191-bf3766f9baca None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:37.647078 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3acc5c-fc07-4702-9191-bf3766f9baca None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:37.647358 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3acc5c-fc07-4702-9191-bf3766f9baca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:37.648174 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3acc5c-fc07-4702-9191-bf3766f9baca None admin] RBAC: Adding query filter params (role.id=d7133ce917f0431883775551ab086d5d, scope.project.id=01eb74db1c404662890e270313372e72, user.id=be785129233e40e9875390e5de7d77d5) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:37.648486 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3acc5c-fc07-4702-9191-bf3766f9baca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:37.649910 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3acc5c-fc07-4702-9191-bf3766f9baca None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:37.655525 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 565/1130] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:16:37 2022] GET /identity/v3/role_assignments?user.id=be785129233e40e9875390e5de7d77d5&scope.project.id=01eb74db1c404662890e270313372e72&role.id=d7133ce917f0431883775551ab086d5d => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:39.820679 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd69f7-4998-45b0-9b62-d642769e82d9 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:39.821265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd69f7-4998-45b0-9b62-d642769e82d9 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:39.821265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd69f7-4998-45b0-9b62-d642769e82d9 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:39.822599 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 566/1131] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:39.827943 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7bef5-ddb5-4f73-90aa-101883ac6e22 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:39.828442 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7bef5-ddb5-4f73-90aa-101883ac6e22 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:39.828546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7bef5-ddb5-4f73-90aa-101883ac6e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:39.845328 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-bcc7bef5-ddb5-4f73-90aa-101883ac6e22 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:39.846494 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-bcc7bef5-ddb5-4f73-90aa-101883ac6e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:39.890946 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:39.911115 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 566/1132] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:16:39.933692 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd69f7-4998-45b0-9b62-d642769e82d9 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:40.032290 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd69f7-4998-45b0-9b62-d642769e82d9 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:40.033765 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:40.064334 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b7c9f-2d01-4a5a-a06b-a6aad94fdf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:40.065108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b7c9f-2d01-4a5a-a06b-a6aad94fdf53 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:40.065390 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b7c9f-2d01-4a5a-a06b-a6aad94fdf53 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:40.065664 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b7c9f-2d01-4a5a-a06b-a6aad94fdf53 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:40.066829 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b7c9f-2d01-4a5a-a06b-a6aad94fdf53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:40.069248 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b7c9f-2d01-4a5a-a06b-a6aad94fdf53 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:40.070948 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 567/1133] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:16:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:40.076906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7bef5-ddb5-4f73-90aa-101883ac6e22 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:40.170250 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7bef5-ddb5-4f73-90aa-101883ac6e22 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:40.202274 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b10049c-a878-4660-8b21-ff00615826f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:40.202938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b10049c-a878-4660-8b21-ff00615826f8 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:40.203220 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b10049c-a878-4660-8b21-ff00615826f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:40.203491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b10049c-a878-4660-8b21-ff00615826f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:40.204188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b10049c-a878-4660-8b21-ff00615826f8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:40.205607 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b10049c-a878-4660-8b21-ff00615826f8 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:40.321318 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 567/1134] 10.222.0.24 () {60 vars in 1141 bytes} [Fri May 27 03:16:40 2022] POST /identity/v3/projects => generated 311 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:16:42.529780 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5baf2ee-b446-4448-94e2-be64c312f98e None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:42.530279 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5baf2ee-b446-4448-94e2-be64c312f98e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:42.530279 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5baf2ee-b446-4448-94e2-be64c312f98e None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:42.531669 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 568/1135] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:42.537395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ceca7-392a-4271-a06d-0339c3574fe2 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:42.537781 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ceca7-392a-4271-a06d-0339c3574fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:42.537944 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ceca7-392a-4271-a06d-0339c3574fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:42.554326 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-ad2ceca7-392a-4271-a06d-0339c3574fe2 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:42.555538 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ad2ceca7-392a-4271-a06d-0339c3574fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:42.580465 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:42.619807 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 568/1136] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:16:42.644066 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5baf2ee-b446-4448-94e2-be64c312f98e None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:42.743068 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5baf2ee-b446-4448-94e2-be64c312f98e None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:42.775569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f8229a-5c4f-45ec-8839-2ff1771ab3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:42.776341 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8229a-5c4f-45ec-8839-2ff1771ab3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:42.776623 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8229a-5c4f-45ec-8839-2ff1771ab3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:42.776896 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8229a-5c4f-45ec-8839-2ff1771ab3d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:42.778053 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f8229a-5c4f-45ec-8839-2ff1771ab3d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:42.779507 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f8229a-5c4f-45ec-8839-2ff1771ab3d9 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:42.781150 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 569/1137] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:16:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:42.787222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ceca7-392a-4271-a06d-0339c3574fe2 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:42.883260 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ceca7-392a-4271-a06d-0339c3574fe2 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:42.915806 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563ee63-a8ff-4000-adc4-2cb474b8d064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:42.916466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563ee63-a8ff-4000-adc4-2cb474b8d064 None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:42.916747 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563ee63-a8ff-4000-adc4-2cb474b8d064 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:42.917022 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563ee63-a8ff-4000-adc4-2cb474b8d064 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:42.917723 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563ee63-a8ff-4000-adc4-2cb474b8d064 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:42.919207 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563ee63-a8ff-4000-adc4-2cb474b8d064 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:43.005789 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 569/1138] 10.222.0.24 () {60 vars in 1132 bytes} [Fri May 27 03:16:42 2022] POST /identity/v3/users => generated 286 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:16:45.225187 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c2349-cd58-4f6b-8413-ca7afc8300c7 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:45.225707 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c2349-cd58-4f6b-8413-ca7afc8300c7 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:45.225798 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c2349-cd58-4f6b-8413-ca7afc8300c7 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:45.227135 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 570/1139] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:45.232399 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306feb0-974e-4533-abe8-61ebba525a65 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:45.232813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306feb0-974e-4533-abe8-61ebba525a65 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:45.232941 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306feb0-974e-4533-abe8-61ebba525a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:45.249163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-2306feb0-974e-4533-abe8-61ebba525a65 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:45.250323 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-2306feb0-974e-4533-abe8-61ebba525a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:45.263245 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:45.312864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 570/1140] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:16:45.336298 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c2349-cd58-4f6b-8413-ca7afc8300c7 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:45.376384 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:45.433782 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c2349-cd58-4f6b-8413-ca7afc8300c7 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:45.466400 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec846a-c88f-4b6b-b878-3c05fedbe3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:45.476052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec846a-c88f-4b6b-b878-3c05fedbe3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:45.476052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec846a-c88f-4b6b-b878-3c05fedbe3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:45.476052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec846a-c88f-4b6b-b878-3c05fedbe3a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:45.478655 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec846a-c88f-4b6b-b878-3c05fedbe3a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:16:45.486441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-ddec846a-c88f-4b6b-b878-3c05fedbe3a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:16:45.486441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:45.486441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:45.486441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:45.486441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:45.486441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:45.486441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:45.486441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:45.486441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:45.486441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:45.486441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:45.486441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:45.486441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:16:45.486441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:16:45.486441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:16:45.486441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:16:45.487579 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:16:45.487579 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:16:45.487579 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:16:45.487579 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:16:45.487579 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:45.487579 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:45.487579 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:45.487579 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:45.487579 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:45.487579 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:45.487579 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:45.487579 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:16:45.487579 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:45.487579 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:45.487579 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:45.487579 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:45.488679 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:45.488679 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:45.488679 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:45.488679 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:45.488679 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:16:45.488679 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:16:45.488679 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:16:45.488679 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:16:45.488679 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:16:45.488679 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:16:45.488679 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:16:45.488679 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:45.488679 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 571/1141] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:16:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:16:45.493803 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306feb0-974e-4533-abe8-61ebba525a65 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:45.587880 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306feb0-974e-4533-abe8-61ebba525a65 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:45.619794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf1ff3-46c2-4604-8d95-a47cff44e53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:45.620441 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf1ff3-46c2-4604-8d95-a47cff44e53d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:45.620716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf1ff3-46c2-4604-8d95-a47cff44e53d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:45.620984 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf1ff3-46c2-4604-8d95-a47cff44e53d None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:45.621711 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf1ff3-46c2-4604-8d95-a47cff44e53d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:45.622012 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf1ff3-46c2-4604-8d95-a47cff44e53d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:45.623389 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf1ff3-46c2-4604-8d95-a47cff44e53d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:45.631038 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 571/1142] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:16:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:45.636722 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec846a-c88f-4b6b-b878-3c05fedbe3a2 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:45.729457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:45.733095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec846a-c88f-4b6b-b878-3c05fedbe3a2 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:45.765138 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15966195-8230-419e-bdba-f3d8ec518aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:45.765993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15966195-8230-419e-bdba-f3d8ec518aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:45.766274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15966195-8230-419e-bdba-f3d8ec518aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:45.766548 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15966195-8230-419e-bdba-f3d8ec518aa7 None admin] PATH_INFO: `/v3/users/eae5418d510e44298f030b5ec540b43e` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:45.780343 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15966195-8230-419e-bdba-f3d8ec518aa7 None admin] RBAC: Authorizing `identity:get_user(user_id=eae5418d510e44298f030b5ec540b43e)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:45.781769 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15966195-8230-419e-bdba-f3d8ec518aa7 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:45.783436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 572/1143] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:16:45 2022] GET /identity/v3/users/eae5418d510e44298f030b5ec540b43e => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:45.789497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf1ff3-46c2-4604-8d95-a47cff44e53d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:45.801699 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:45.883513 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf1ff3-46c2-4604-8d95-a47cff44e53d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:45.914897 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51319250-e2d1-49dd-80ea-5cf8b51027c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:45.915520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51319250-e2d1-49dd-80ea-5cf8b51027c9 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:45.915795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51319250-e2d1-49dd-80ea-5cf8b51027c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:45.916062 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51319250-e2d1-49dd-80ea-5cf8b51027c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:45.979856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51319250-e2d1-49dd-80ea-5cf8b51027c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:45.981304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51319250-e2d1-49dd-80ea-5cf8b51027c9 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:45.991657 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:46.073052 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 572/1144] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:16:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:16:46.079147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15966195-8230-419e-bdba-f3d8ec518aa7 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:46.176426 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15966195-8230-419e-bdba-f3d8ec518aa7 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:46.209295 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba4bc5-2ed6-4d15-8d96-75f873d713ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:46.210132 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba4bc5-2ed6-4d15-8d96-75f873d713ae None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:46.210411 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba4bc5-2ed6-4d15-8d96-75f873d713ae None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:46.210683 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba4bc5-2ed6-4d15-8d96-75f873d713ae None admin] PATH_INFO: `/v3/projects/a767ca92f1a1423c9c049078e4971adb` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:46.211878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba4bc5-2ed6-4d15-8d96-75f873d713ae None admin] RBAC: Authorizing `identity:get_project(project_id=a767ca92f1a1423c9c049078e4971adb)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:46.213304 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba4bc5-2ed6-4d15-8d96-75f873d713ae None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:46.214991 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 573/1145] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:16:46 2022] GET /identity/v3/projects/a767ca92f1a1423c9c049078e4971adb => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:46.220807 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51319250-e2d1-49dd-80ea-5cf8b51027c9 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:46.313406 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51319250-e2d1-49dd-80ea-5cf8b51027c9 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:46.323953 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:46.345665 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fcb0b6-cda6-493a-9580-e1d6130807c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:46.346293 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fcb0b6-cda6-493a-9580-e1d6130807c4 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:46.356913 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fcb0b6-cda6-493a-9580-e1d6130807c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:46.356913 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fcb0b6-cda6-493a-9580-e1d6130807c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:46.356913 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fcb0b6-cda6-493a-9580-e1d6130807c4 None admin] RBAC: Adding query filter params (role.id=e5a908bd3ed5445b82a6053246533b88, scope.project.id=a767ca92f1a1423c9c049078e4971adb, user.id=eae5418d510e44298f030b5ec540b43e) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:46.356913 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fcb0b6-cda6-493a-9580-e1d6130807c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:46.356913 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fcb0b6-cda6-493a-9580-e1d6130807c4 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:46.357317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 573/1146] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:16:46 2022] GET /identity/v3/role_assignments?user.id=eae5418d510e44298f030b5ec540b43e&scope.project.id=a767ca92f1a1423c9c049078e4971adb&role.id=e5a908bd3ed5445b82a6053246533b88 => generated 259 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:48.511321 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f8140-eb18-4f6a-b10c-01f810466bb3 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:48.511843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f8140-eb18-4f6a-b10c-01f810466bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:48.511843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f8140-eb18-4f6a-b10c-01f810466bb3 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:48.513129 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 574/1147] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:48.518387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181e2f5-09b6-402a-966f-56c8c74d61de None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:48.518811 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181e2f5-09b6-402a-966f-56c8c74d61de None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:48.518971 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181e2f5-09b6-402a-966f-56c8c74d61de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:48.535335 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-9181e2f5-09b6-402a-966f-56c8c74d61de None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:48.536513 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-9181e2f5-09b6-402a-966f-56c8c74d61de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:48.599907 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 574/1148] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:16:48.622231 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f8140-eb18-4f6a-b10c-01f810466bb3 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:48.647009 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:48.719218 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f8140-eb18-4f6a-b10c-01f810466bb3 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:48.751756 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce46ec-fd10-4c6d-81ff-e80ba6f5ae29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:48.752535 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce46ec-fd10-4c6d-81ff-e80ba6f5ae29 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:48.752814 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce46ec-fd10-4c6d-81ff-e80ba6f5ae29 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:48.753085 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ce46ec-fd10-4c6d-81ff-e80ba6f5ae29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:48.763412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ce46ec-fd10-4c6d-81ff-e80ba6f5ae29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:16:48.770999 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-54ce46ec-fd10-4c6d-81ff-e80ba6f5ae29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:16:48.770999 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:48.770999 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:48.770999 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:48.770999 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:48.770999 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:48.770999 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:48.770999 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:48.770999 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:48.770999 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:48.770999 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:48.770999 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:48.770999 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:16:48.770999 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:16:48.770999 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:16:48.770999 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:16:48.772086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:16:48.772086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:16:48.772086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:16:48.772086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:16:48.772086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:48.772086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:48.772086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:48.772086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:48.772086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:48.772086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:48.772086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:48.772086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:16:48.772086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:48.772086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:48.772086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:48.772086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:48.773115 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:48.773115 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:48.773115 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:48.773115 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:48.773115 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:16:48.773115 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:16:48.773115 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:16:48.773115 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:16:48.773115 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:16:48.773115 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:16:48.773115 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:16:48.773115 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:48.773115 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 575/1149] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:16:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:16:48.778192 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181e2f5-09b6-402a-966f-56c8c74d61de None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:48.872613 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181e2f5-09b6-402a-966f-56c8c74d61de None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:48.874798 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:48.904356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a33a1b-14ea-4071-a0e2-e82755f4887a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:48.905014 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a33a1b-14ea-4071-a0e2-e82755f4887a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:48.905298 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a33a1b-14ea-4071-a0e2-e82755f4887a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:48.905574 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a33a1b-14ea-4071-a0e2-e82755f4887a None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:48.906310 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a33a1b-14ea-4071-a0e2-e82755f4887a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:48.906618 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a33a1b-14ea-4071-a0e2-e82755f4887a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:48.908007 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a33a1b-14ea-4071-a0e2-e82755f4887a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:48.916211 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 575/1150] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:16:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:48.921950 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce46ec-fd10-4c6d-81ff-e80ba6f5ae29 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:49.004571 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:49.019555 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ce46ec-fd10-4c6d-81ff-e80ba6f5ae29 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:49.053109 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632438a-588e-475d-bbc7-71c07fa249ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:49.053961 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632438a-588e-475d-bbc7-71c07fa249ee None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:49.054242 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632438a-588e-475d-bbc7-71c07fa249ee None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:49.054516 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632438a-588e-475d-bbc7-71c07fa249ee None admin] PATH_INFO: `/v3/users/eae5418d510e44298f030b5ec540b43e` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:49.055738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1632438a-588e-475d-bbc7-71c07fa249ee None admin] RBAC: Authorizing `identity:get_user(user_id=eae5418d510e44298f030b5ec540b43e)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:49.057170 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1632438a-588e-475d-bbc7-71c07fa249ee None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:49.058770 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 576/1151] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:16:48 2022] GET /identity/v3/users/eae5418d510e44298f030b5ec540b43e => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:49.064369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a33a1b-14ea-4071-a0e2-e82755f4887a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:49.161067 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a33a1b-14ea-4071-a0e2-e82755f4887a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:49.193405 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcaf67c-0714-4a3a-abb6-cd8d936ece5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:49.194254 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcaf67c-0714-4a3a-abb6-cd8d936ece5a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:49.194536 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcaf67c-0714-4a3a-abb6-cd8d936ece5a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:49.194815 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcaf67c-0714-4a3a-abb6-cd8d936ece5a None admin] PATH_INFO: `/v3/projects/a767ca92f1a1423c9c049078e4971adb` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:49.196010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcaf67c-0714-4a3a-abb6-cd8d936ece5a None admin] RBAC: Authorizing `identity:get_project(project_id=a767ca92f1a1423c9c049078e4971adb)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:49.197466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcaf67c-0714-4a3a-abb6-cd8d936ece5a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:49.199428 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 576/1152] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:16:49 2022] GET /identity/v3/projects/a767ca92f1a1423c9c049078e4971adb => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:49.205190 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632438a-588e-475d-bbc7-71c07fa249ee None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:49.311980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632438a-588e-475d-bbc7-71c07fa249ee None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:49.344884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3848cf-84b0-42d2-8d14-9f8c1300b05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:49.345761 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3848cf-84b0-42d2-8d14-9f8c1300b05b None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:49.346041 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3848cf-84b0-42d2-8d14-9f8c1300b05b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:49.346312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3848cf-84b0-42d2-8d14-9f8c1300b05b None admin] PATH_INFO: `/v3/projects/a767ca92f1a1423c9c049078e4971adb/users/eae5418d510e44298f030b5ec540b43e/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:49.348834 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3848cf-84b0-42d2-8d14-9f8c1300b05b None admin] RBAC: Authorizing `identity:create_grant(project_id=a767ca92f1a1423c9c049078e4971adb, user_id=eae5418d510e44298f030b5ec540b43e, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:49.350256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3848cf-84b0-42d2-8d14-9f8c1300b05b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:49.427588 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 577/1153] 10.222.0.24 () {58 vars in 1439 bytes} [Fri May 27 03:16:49 2022] PUT /identity/v3/projects/a767ca92f1a1423c9c049078e4971adb/users/eae5418d510e44298f030b5ec540b43e/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:16:51.578349 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48f56b-f3f2-4717-908e-2e474e90d6ed None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:51.578956 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48f56b-f3f2-4717-908e-2e474e90d6ed None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:51.578956 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48f56b-f3f2-4717-908e-2e474e90d6ed None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:51.580279 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 577/1154] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:51.586039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d5eb2-ce6e-4bb4-92ea-016d31bcac26 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:51.586443 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d5eb2-ce6e-4bb4-92ea-016d31bcac26 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:51.586603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d5eb2-ce6e-4bb4-92ea-016d31bcac26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:51.603490 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-d62d5eb2-ce6e-4bb4-92ea-016d31bcac26 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:51.604661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d62d5eb2-ce6e-4bb4-92ea-016d31bcac26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:51.612385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:51.673431 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 578/1155] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:16:51.696290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48f56b-f3f2-4717-908e-2e474e90d6ed None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:51.731190 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:51.793802 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48f56b-f3f2-4717-908e-2e474e90d6ed None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:51.825179 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380600b-cebf-4bb2-a233-eb45d248701d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:51.825952 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380600b-cebf-4bb2-a233-eb45d248701d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:51.826226 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380600b-cebf-4bb2-a233-eb45d248701d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:51.826494 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380600b-cebf-4bb2-a233-eb45d248701d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:51.836594 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b380600b-cebf-4bb2-a233-eb45d248701d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:16:51.843976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-b380600b-cebf-4bb2-a233-eb45d248701d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:16:51.843976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:51.843976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:51.843976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:51.843976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:51.843976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:51.843976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:51.843976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:51.843976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:51.843976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:51.843976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:51.843976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:51.843976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:16:51.843976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:16:51.843976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:16:51.843976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:16:51.845118 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:16:51.845118 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:16:51.845118 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:16:51.845118 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:16:51.845118 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:51.845118 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:51.845118 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:51.845118 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:51.845118 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:51.845118 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:51.845118 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:51.845118 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:16:51.845118 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:51.845118 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:51.845118 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:51.845118 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:51.846265 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:51.846265 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:51.846265 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:51.846265 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:51.846265 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:16:51.846265 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:16:51.846265 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:16:51.846265 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:16:51.846265 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:16:51.846265 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:16:51.846265 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:16:51.846265 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:16:51.846265 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 578/1156] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:16:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:16:51.851423 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d5eb2-ce6e-4bb4-92ea-016d31bcac26 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:51.970575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d5eb2-ce6e-4bb4-92ea-016d31bcac26 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:52.003380 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba8806-f919-464c-b943-458c8905c3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:52.004558 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba8806-f919-464c-b943-458c8905c3ee None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:52.004837 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba8806-f919-464c-b943-458c8905c3ee None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:52.005105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba8806-f919-464c-b943-458c8905c3ee None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:52.005836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba8806-f919-464c-b943-458c8905c3ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:52.006133 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba8806-f919-464c-b943-458c8905c3ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:52.007507 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba8806-f919-464c-b943-458c8905c3ee None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:52.015332 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 579/1157] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:16:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:52.021138 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380600b-cebf-4bb2-a233-eb45d248701d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:52.110816 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:52.117655 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380600b-cebf-4bb2-a233-eb45d248701d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:52.149267 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fd816-a829-4533-9896-e3027a2f1887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:52.150120 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fd816-a829-4533-9896-e3027a2f1887 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:52.150396 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fd816-a829-4533-9896-e3027a2f1887 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:52.150664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fd816-a829-4533-9896-e3027a2f1887 None admin] PATH_INFO: `/v3/users/eae5418d510e44298f030b5ec540b43e` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:52.151859 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99fd816-a829-4533-9896-e3027a2f1887 None admin] RBAC: Authorizing `identity:get_user(user_id=eae5418d510e44298f030b5ec540b43e)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:52.153281 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99fd816-a829-4533-9896-e3027a2f1887 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:52.154945 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 579/1158] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:16:52 2022] GET /identity/v3/users/eae5418d510e44298f030b5ec540b43e => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:52.161129 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba8806-f919-464c-b943-458c8905c3ee None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:52.164815 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:52.269656 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba8806-f919-464c-b943-458c8905c3ee None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:52.301782 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0486420-3908-4532-853f-820124f79078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:52.302413 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0486420-3908-4532-853f-820124f79078 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:52.302688 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0486420-3908-4532-853f-820124f79078 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:52.302987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0486420-3908-4532-853f-820124f79078 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:52.366395 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0486420-3908-4532-853f-820124f79078 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:52.367872 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0486420-3908-4532-853f-820124f79078 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:52.370351 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:52.461147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 580/1159] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:16:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:16:52.467005 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fd816-a829-4533-9896-e3027a2f1887 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:52.563716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fd816-a829-4533-9896-e3027a2f1887 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:52.598047 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b558dca-02f7-4fc3-8ab2-733b1136c523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:52.598923 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b558dca-02f7-4fc3-8ab2-733b1136c523 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:52.599203 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b558dca-02f7-4fc3-8ab2-733b1136c523 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:52.599475 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b558dca-02f7-4fc3-8ab2-733b1136c523 None admin] PATH_INFO: `/v3/projects/a767ca92f1a1423c9c049078e4971adb` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:52.600753 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b558dca-02f7-4fc3-8ab2-733b1136c523 None admin] RBAC: Authorizing `identity:get_project(project_id=a767ca92f1a1423c9c049078e4971adb)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:52.602202 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b558dca-02f7-4fc3-8ab2-733b1136c523 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:52.603946 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 580/1160] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:16:52 2022] GET /identity/v3/projects/a767ca92f1a1423c9c049078e4971adb => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:52.610035 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0486420-3908-4532-853f-820124f79078 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:52.705494 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0486420-3908-4532-853f-820124f79078 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:52.707695 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:52.737860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7809b1a-fee3-4c7f-8ef4-aeec2232fc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:52.739094 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7809b1a-fee3-4c7f-8ef4-aeec2232fc1c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:52.739379 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7809b1a-fee3-4c7f-8ef4-aeec2232fc1c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:52.739654 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7809b1a-fee3-4c7f-8ef4-aeec2232fc1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:52.740495 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7809b1a-fee3-4c7f-8ef4-aeec2232fc1c None admin] RBAC: Adding query filter params (role.id=e5a908bd3ed5445b82a6053246533b88, scope.project.id=a767ca92f1a1423c9c049078e4971adb, user.id=eae5418d510e44298f030b5ec540b43e) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:52.740857 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7809b1a-fee3-4c7f-8ef4-aeec2232fc1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:52.742409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7809b1a-fee3-4c7f-8ef4-aeec2232fc1c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:52.748622 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 581/1161] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:16:52 2022] GET /identity/v3/role_assignments?user.id=eae5418d510e44298f030b5ec540b43e&scope.project.id=a767ca92f1a1423c9c049078e4971adb&role.id=e5a908bd3ed5445b82a6053246533b88 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:54.932497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43ed04-6e74-43c6-9673-6aad64fc946e None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:54.933044 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43ed04-6e74-43c6-9673-6aad64fc946e None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:54.933044 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43ed04-6e74-43c6-9673-6aad64fc946e None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:54.934395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 581/1162] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:54.939659 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7addb0cc-96e2-494c-81f4-416ec44a4684 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:54.940054 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7addb0cc-96e2-494c-81f4-416ec44a4684 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:54.940205 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7addb0cc-96e2-494c-81f4-416ec44a4684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:54.957720 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-7addb0cc-96e2-494c-81f4-416ec44a4684 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:54.958907 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-7addb0cc-96e2-494c-81f4-416ec44a4684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:55.022131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 582/1163] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:16:55.043822 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43ed04-6e74-43c6-9673-6aad64fc946e None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:55.073394 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:55.143048 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43ed04-6e74-43c6-9673-6aad64fc946e None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:55.175097 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0fafa5-b5e4-445f-a21f-5d6bc225e533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:55.175752 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0fafa5-b5e4-445f-a21f-5d6bc225e533 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:55.176034 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0fafa5-b5e4-445f-a21f-5d6bc225e533 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:55.176308 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0fafa5-b5e4-445f-a21f-5d6bc225e533 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:55.239797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0fafa5-b5e4-445f-a21f-5d6bc225e533 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:55.241167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0fafa5-b5e4-445f-a21f-5d6bc225e533 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:55.267970 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:55.336040 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 582/1164] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:16:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:16:55.342025 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7addb0cc-96e2-494c-81f4-416ec44a4684 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:55.433227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:55.436872 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7addb0cc-96e2-494c-81f4-416ec44a4684 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:55.470063 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f608e6d-c32f-4148-a379-61a7059615cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:55.470866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f608e6d-c32f-4148-a379-61a7059615cb None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:55.471154 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f608e6d-c32f-4148-a379-61a7059615cb None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:55.471424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f608e6d-c32f-4148-a379-61a7059615cb None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:55.477930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f608e6d-c32f-4148-a379-61a7059615cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:55.479402 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f608e6d-c32f-4148-a379-61a7059615cb None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:55.493701 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-0f608e6d-c32f-4148-a379-61a7059615cb None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 27 03:16:55.493701 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:55.493701 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 27 03:16:55.493701 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 27 03:16:55.493701 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:16:55.493701 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:16:55.493701 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:16:55.493701 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:16:55.493701 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 27 03:16:55.493701 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:55.493701 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:16:55.493701 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:55.493701 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:16:55.493701 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:16:55.493701 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:16:55.493701 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:16:55.493701 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:16:55.494890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:16:55.494890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:16:55.494890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:16:55.494890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:16:55.494890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:16:55.494890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:16:55.494890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:16:55.494890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:16:55.494890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:16:55.494890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:16:55.494890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:16:55.494890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:16:55.494890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:16:55.494890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:16:55.494890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:16:55.494890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:16:55.495920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:16:55.495920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:16:55.495920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:16:55.495920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:16:55.495920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:16:55.495920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:16:55.495920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:16:55.495920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:16:55.495920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:16:55.495920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:16:55.495920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 27 03:16:55.495920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:16:55.495920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 27 03:16:55.495920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:16:55.495920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 583/1165] 10.222.0.24 () {56 vars in 1117 bytes} [Fri May 27 03:16:55 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:16:55.500727 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0fafa5-b5e4-445f-a21f-5d6bc225e533 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:55.595619 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0fafa5-b5e4-445f-a21f-5d6bc225e533 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:55.622568 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:55.628253 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a66d33-0451-4aee-9e31-a6d5bedc57e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:55.629447 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a66d33-0451-4aee-9e31-a6d5bedc57e5 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:55.629731 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a66d33-0451-4aee-9e31-a6d5bedc57e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:55.630004 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a66d33-0451-4aee-9e31-a6d5bedc57e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:55.630723 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a66d33-0451-4aee-9e31-a6d5bedc57e5 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:16:55.631063 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a66d33-0451-4aee-9e31-a6d5bedc57e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:55.632434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a66d33-0451-4aee-9e31-a6d5bedc57e5 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:55.639439 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 583/1166] 10.222.0.24 () {56 vars in 1115 bytes} [Fri May 27 03:16:55 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:16:57.814048 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ea4dc-fbb4-45f4-92c6-83e068a39ecf None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:57.814573 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ea4dc-fbb4-45f4-92c6-83e068a39ecf None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:57.814573 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ea4dc-fbb4-45f4-92c6-83e068a39ecf None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:57.816051 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 584/1167] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:57.821299 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0674a-9579-481a-806c-3dcd57eea317 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:57.821724 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0674a-9579-481a-806c-3dcd57eea317 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:57.821863 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0674a-9579-481a-806c-3dcd57eea317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:57.838851 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d0b0674a-9579-481a-806c-3dcd57eea317 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:57.840040 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d0b0674a-9579-481a-806c-3dcd57eea317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:57.903417 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 584/1168] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:16:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:16:57.925921 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ea4dc-fbb4-45f4-92c6-83e068a39ecf None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:58.028315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ea4dc-fbb4-45f4-92c6-83e068a39ecf None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:58.060256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f8319-eeea-4b57-9cd1-0a55c80a72f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:58.060904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f8319-eeea-4b57-9cd1-0a55c80a72f6 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:58.061187 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f8319-eeea-4b57-9cd1-0a55c80a72f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:58.061462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f8319-eeea-4b57-9cd1-0a55c80a72f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:58.062206 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f8319-eeea-4b57-9cd1-0a55c80a72f6 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:58.063628 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f8319-eeea-4b57-9cd1-0a55c80a72f6 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:58.142977 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 585/1169] 10.222.0.24 () {60 vars in 1138 bytes} [Fri May 27 03:16:57 2022] POST /identity/v3/domains => generated 267 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:17:00.329152 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16addf1-f559-4cac-bc91-925a134b53a6 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:00.329654 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16addf1-f559-4cac-bc91-925a134b53a6 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:00.329654 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16addf1-f559-4cac-bc91-925a134b53a6 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:00.331039 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 585/1170] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:17:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:00.336840 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fff3d-0903-46ef-adfe-aa789e364f3d None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:00.337258 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fff3d-0903-46ef-adfe-aa789e364f3d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:00.337389 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fff3d-0903-46ef-adfe-aa789e364f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:00.353613 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-ef2fff3d-0903-46ef-adfe-aa789e364f3d None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:00.354783 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ef2fff3d-0903-46ef-adfe-aa789e364f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:00.367334 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:17:00.418623 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 586/1171] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:17:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:17:00.442492 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16addf1-f559-4cac-bc91-925a134b53a6 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:00.539232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16addf1-f559-4cac-bc91-925a134b53a6 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:00.561964 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:17:00.574438 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8faea6-47a2-44b9-9dab-074cda6f70a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:00.578477 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8faea6-47a2-44b9-9dab-074cda6f70a1 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:00.578477 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8faea6-47a2-44b9-9dab-074cda6f70a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:00.578477 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8faea6-47a2-44b9-9dab-074cda6f70a1 None admin] PATH_INFO: `/v3/domains/c76df64daae344bf98d489408603aa71` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:00.586326 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8faea6-47a2-44b9-9dab-074cda6f70a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=c76df64daae344bf98d489408603aa71)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:00.587721 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8faea6-47a2-44b9-9dab-074cda6f70a1 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:00.589371 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 586/1172] 10.222.0.24 () {56 vars in 1183 bytes} [Fri May 27 03:17:00 2022] GET /identity/v3/domains/c76df64daae344bf98d489408603aa71 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:17:00.596736 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fff3d-0903-46ef-adfe-aa789e364f3d None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:00.691488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fff3d-0903-46ef-adfe-aa789e364f3d None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:00.723943 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814eb6d-1eb5-40ec-916e-664dbc017121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:00.724583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814eb6d-1eb5-40ec-916e-664dbc017121 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:00.724866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814eb6d-1eb5-40ec-916e-664dbc017121 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:00.725141 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814eb6d-1eb5-40ec-916e-664dbc017121 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:00.725846 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814eb6d-1eb5-40ec-916e-664dbc017121 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:00.727306 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814eb6d-1eb5-40ec-916e-664dbc017121 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:00.814270 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:17:00.820700 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 587/1173] 10.222.0.24 () {60 vars in 1141 bytes} [Fri May 27 03:17:00 2022] POST /identity/v3/projects => generated 361 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:17:03.023850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be8436-5db8-4c7e-9369-b7fc90fa0347 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:03.024421 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be8436-5db8-4c7e-9369-b7fc90fa0347 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:03.024421 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be8436-5db8-4c7e-9369-b7fc90fa0347 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:03.025750 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 587/1174] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:03.031104 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c97e3-c0d8-4eae-aa51-570ffd9a48fe None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:03.031476 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c97e3-c0d8-4eae-aa51-570ffd9a48fe None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:03.031677 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c97e3-c0d8-4eae-aa51-570ffd9a48fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:03.049006 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-c02c97e3-c0d8-4eae-aa51-570ffd9a48fe None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:03.050220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c02c97e3-c0d8-4eae-aa51-570ffd9a48fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:03.115437 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 588/1175] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:17:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:17:03.140258 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be8436-5db8-4c7e-9369-b7fc90fa0347 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:03.237101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be8436-5db8-4c7e-9369-b7fc90fa0347 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:03.269538 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9caa17-c3c0-4793-851e-c45b193925cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:03.270384 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9caa17-c3c0-4793-851e-c45b193925cc None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:03.270660 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9caa17-c3c0-4793-851e-c45b193925cc None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:03.270963 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9caa17-c3c0-4793-851e-c45b193925cc None admin] PATH_INFO: `/v3/domains/c76df64daae344bf98d489408603aa71` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:03.272130 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9caa17-c3c0-4793-851e-c45b193925cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=c76df64daae344bf98d489408603aa71)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:03.273553 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9caa17-c3c0-4793-851e-c45b193925cc None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:03.275211 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 588/1176] 10.222.0.24 () {56 vars in 1183 bytes} [Fri May 27 03:17:03 2022] GET /identity/v3/domains/c76df64daae344bf98d489408603aa71 => generated 267 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:17:03.280713 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c97e3-c0d8-4eae-aa51-570ffd9a48fe None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:03.387322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c97e3-c0d8-4eae-aa51-570ffd9a48fe None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:03.415350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:17:03.421005 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1429ad-1712-4917-8737-b4752be9d08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:03.421665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1429ad-1712-4917-8737-b4752be9d08f None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:03.429463 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1429ad-1712-4917-8737-b4752be9d08f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:03.429463 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1429ad-1712-4917-8737-b4752be9d08f None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:03.429463 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1429ad-1712-4917-8737-b4752be9d08f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:03.429463 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1429ad-1712-4917-8737-b4752be9d08f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:03.510123 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 589/1177] 10.222.0.24 () {60 vars in 1132 bytes} [Fri May 27 03:17:03 2022] POST /identity/v3/users => generated 311 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:17:05.714317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70e927-8377-42f4-8925-a7e1f84f730d None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:05.714871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70e927-8377-42f4-8925-a7e1f84f730d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:05.714871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb70e927-8377-42f4-8925-a7e1f84f730d None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:05.716253 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 589/1178] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:05.721628 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe03c57-9e93-4a43-bca4-c94aac347cc7 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:05.722020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe03c57-9e93-4a43-bca4-c94aac347cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:05.722206 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe03c57-9e93-4a43-bca4-c94aac347cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:05.738999 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-1fe03c57-9e93-4a43-bca4-c94aac347cc7 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:05.740165 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1fe03c57-9e93-4a43-bca4-c94aac347cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:05.804557 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 590/1179] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:17:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:17:05.827572 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70e927-8377-42f4-8925-a7e1f84f730d None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:05.879417 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:17:05.925164 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb70e927-8377-42f4-8925-a7e1f84f730d None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:05.956961 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4a4f2-3921-47d7-b163-4d9d85bf6f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:05.957726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4a4f2-3921-47d7-b163-4d9d85bf6f50 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:05.958004 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4a4f2-3921-47d7-b163-4d9d85bf6f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:05.958273 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d4a4f2-3921-47d7-b163-4d9d85bf6f50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:05.968057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d4a4f2-3921-47d7-b163-4d9d85bf6f50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:17:05.975530 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-12d4a4f2-3921-47d7-b163-4d9d85bf6f50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:17:05.975530 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:17:05.975530 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:17:05.975530 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:17:05.975530 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:17:05.975530 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:17:05.975530 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:17:05.975530 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:17:05.975530 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:17:05.975530 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:17:05.975530 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:17:05.975530 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:17:05.975530 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:17:05.975530 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:17:05.975530 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:17:05.975530 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:17:05.977380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:17:05.977380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:17:05.977380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:17:05.977380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:17:05.977380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:17:05.977380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:17:05.977380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:17:05.977380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:17:05.977380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:17:05.977380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:17:05.977380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:17:05.977380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:17:05.977380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:17:05.977380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:17:05.977380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:17:05.977380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:17:05.978417 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:17:05.978417 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:17:05.978417 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:17:05.978417 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:17:05.978417 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:17:05.978417 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:17:05.978417 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:17:05.978417 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:17:05.978417 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:17:05.978417 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:17:05.978417 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:17:05.978417 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:17:05.978417 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 590/1180] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:17:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:17:05.982895 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe03c57-9e93-4a43-bca4-c94aac347cc7 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:06.077668 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe03c57-9e93-4a43-bca4-c94aac347cc7 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:06.093657 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:17:06.111016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b59b1-0afb-4a3b-8058-8578e02900c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:06.111676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b59b1-0afb-4a3b-8058-8578e02900c0 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:06.111956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b59b1-0afb-4a3b-8058-8578e02900c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:06.112248 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b59b1-0afb-4a3b-8058-8578e02900c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:06.113516 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955b59b1-0afb-4a3b-8058-8578e02900c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:17:06.113819 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955b59b1-0afb-4a3b-8058-8578e02900c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:06.115205 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955b59b1-0afb-4a3b-8058-8578e02900c0 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:06.123083 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 591/1181] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:17:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:17:06.128582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4a4f2-3921-47d7-b163-4d9d85bf6f50 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:06.224544 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d4a4f2-3921-47d7-b163-4d9d85bf6f50 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:06.236493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:17:06.256484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d8e77e-da0a-49c5-8d41-1bdf63d27c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:06.257333 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d8e77e-da0a-49c5-8d41-1bdf63d27c5e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:06.257608 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d8e77e-da0a-49c5-8d41-1bdf63d27c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:06.257874 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d8e77e-da0a-49c5-8d41-1bdf63d27c5e None admin] PATH_INFO: `/v3/users/01a15cf338a64e579a964fbb770410fe` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:06.270770 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d8e77e-da0a-49c5-8d41-1bdf63d27c5e None admin] RBAC: Authorizing `identity:get_user(user_id=01a15cf338a64e579a964fbb770410fe)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:06.272301 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d8e77e-da0a-49c5-8d41-1bdf63d27c5e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:06.274243 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 591/1182] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:17:06 2022] GET /identity/v3/users/01a15cf338a64e579a964fbb770410fe => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:17:06.279610 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b59b1-0afb-4a3b-8058-8578e02900c0 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:06.375025 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b59b1-0afb-4a3b-8058-8578e02900c0 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:06.407628 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53e3f3-7584-43f8-ba0d-858f184b50e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:06.408257 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53e3f3-7584-43f8-ba0d-858f184b50e5 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:06.408534 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53e3f3-7584-43f8-ba0d-858f184b50e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:06.408805 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53e3f3-7584-43f8-ba0d-858f184b50e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:06.451667 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:17:06.472408 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f53e3f3-7584-43f8-ba0d-858f184b50e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:06.473826 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f53e3f3-7584-43f8-ba0d-858f184b50e5 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:06.567571 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 592/1183] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:17:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:17:06.573294 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d8e77e-da0a-49c5-8d41-1bdf63d27c5e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:06.667659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d8e77e-da0a-49c5-8d41-1bdf63d27c5e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:06.699124 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf741336-a274-44eb-ab90-09e3b2b832f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:06.699959 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf741336-a274-44eb-ab90-09e3b2b832f6 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:06.700236 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf741336-a274-44eb-ab90-09e3b2b832f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:06.700507 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf741336-a274-44eb-ab90-09e3b2b832f6 None admin] PATH_INFO: `/v3/projects/647004410b244685b36a7d774ed53f18` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:06.701647 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf741336-a274-44eb-ab90-09e3b2b832f6 None admin] RBAC: Authorizing `identity:get_project(project_id=647004410b244685b36a7d774ed53f18)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:06.703072 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf741336-a274-44eb-ab90-09e3b2b832f6 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:06.704701 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 592/1184] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:17:06 2022] GET /identity/v3/projects/647004410b244685b36a7d774ed53f18 => generated 361 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:17:06.711044 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53e3f3-7584-43f8-ba0d-858f184b50e5 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:06.783964 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:17:06.804250 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53e3f3-7584-43f8-ba0d-858f184b50e5 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:06.836744 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208828e-61bd-430a-8363-237a78634cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:06.837319 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208828e-61bd-430a-8363-237a78634cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:06.837597 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208828e-61bd-430a-8363-237a78634cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:06.837867 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208828e-61bd-430a-8363-237a78634cf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:06.838677 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208828e-61bd-430a-8363-237a78634cf4 None admin] RBAC: Adding query filter params (role.id=e5a908bd3ed5445b82a6053246533b88, scope.project.id=647004410b244685b36a7d774ed53f18, user.id=01a15cf338a64e579a964fbb770410fe) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:17:06.839013 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208828e-61bd-430a-8363-237a78634cf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:06.840414 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208828e-61bd-430a-8363-237a78634cf4 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:06.845590 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 593/1185] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:17:06 2022] GET /identity/v3/role_assignments?user.id=01a15cf338a64e579a964fbb770410fe&scope.project.id=647004410b244685b36a7d774ed53f18&role.id=e5a908bd3ed5445b82a6053246533b88 => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:17:08.996214 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1205a0a-500d-439e-b1b6-e0f04f09643a None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:08.996792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1205a0a-500d-439e-b1b6-e0f04f09643a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:08.996792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1205a0a-500d-439e-b1b6-e0f04f09643a None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:08.998124 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 593/1186] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:09.003935 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534c1fe-4dd2-4c7a-b5e5-6259cfba78ba None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:09.004333 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534c1fe-4dd2-4c7a-b5e5-6259cfba78ba None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:09.004485 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534c1fe-4dd2-4c7a-b5e5-6259cfba78ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:09.021012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-1534c1fe-4dd2-4c7a-b5e5-6259cfba78ba None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:09.022189 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1534c1fe-4dd2-4c7a-b5e5-6259cfba78ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:09.087150 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 594/1187] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:17:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:17:09.110040 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1205a0a-500d-439e-b1b6-e0f04f09643a None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:09.150080 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:17:09.207877 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1205a0a-500d-439e-b1b6-e0f04f09643a None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:09.240421 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8712c-05d3-4b07-af2a-7f3f53466e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:09.241188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8712c-05d3-4b07-af2a-7f3f53466e2d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:09.241467 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8712c-05d3-4b07-af2a-7f3f53466e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:09.241738 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8712c-05d3-4b07-af2a-7f3f53466e2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:09.251399 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb8712c-05d3-4b07-af2a-7f3f53466e2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:17:09.258933 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-8cb8712c-05d3-4b07-af2a-7f3f53466e2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:17:09.258933 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:17:09.258933 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:17:09.258933 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:17:09.258933 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:17:09.258933 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:17:09.258933 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:17:09.258933 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:17:09.258933 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:17:09.258933 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:17:09.258933 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:17:09.258933 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:17:09.258933 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:17:09.258933 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:17:09.258933 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:17:09.258933 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:17:09.259994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:17:09.259994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:17:09.259994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:17:09.259994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:17:09.259994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:17:09.259994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:17:09.259994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:17:09.259994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:17:09.259994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:17:09.259994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:17:09.259994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:17:09.259994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:17:09.259994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:17:09.259994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:17:09.259994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:17:09.259994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:17:09.261019 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:17:09.261019 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:17:09.261019 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:17:09.261019 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:17:09.261019 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:17:09.261019 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:17:09.261019 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:17:09.261019 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:17:09.261019 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:17:09.261019 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:17:09.261019 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:17:09.261019 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:17:09.261019 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 594/1188] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:17:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:17:09.265975 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1534c1fe-4dd2-4c7a-b5e5-6259cfba78ba None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:09.333346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:17:09.360733 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1534c1fe-4dd2-4c7a-b5e5-6259cfba78ba None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:09.393120 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc8431-c292-408e-a680-542023684671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:09.393765 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc8431-c292-408e-a680-542023684671 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:09.394044 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc8431-c292-408e-a680-542023684671 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:09.394316 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc8431-c292-408e-a680-542023684671 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:09.395070 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc8431-c292-408e-a680-542023684671 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:17:09.395376 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc8431-c292-408e-a680-542023684671 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:09.396734 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc8431-c292-408e-a680-542023684671 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:09.404575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 595/1189] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:17:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:17:09.410787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8712c-05d3-4b07-af2a-7f3f53466e2d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:09.512136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:17:09.515743 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8712c-05d3-4b07-af2a-7f3f53466e2d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:09.547994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd35a4-eea0-42e5-ae21-2a56204da020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:09.548844 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd35a4-eea0-42e5-ae21-2a56204da020 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:09.549123 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd35a4-eea0-42e5-ae21-2a56204da020 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:09.549394 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd35a4-eea0-42e5-ae21-2a56204da020 None admin] PATH_INFO: `/v3/users/01a15cf338a64e579a964fbb770410fe` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:09.550584 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd35a4-eea0-42e5-ae21-2a56204da020 None admin] RBAC: Authorizing `identity:get_user(user_id=01a15cf338a64e579a964fbb770410fe)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:09.552054 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd35a4-eea0-42e5-ae21-2a56204da020 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:09.553853 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 595/1190] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:17:09 2022] GET /identity/v3/users/01a15cf338a64e579a964fbb770410fe => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:17:09.559526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc8431-c292-408e-a680-542023684671 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:09.655690 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc8431-c292-408e-a680-542023684671 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:09.689047 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c077db-3d2d-409e-9f2a-be9a2cf95bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:09.689893 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c077db-3d2d-409e-9f2a-be9a2cf95bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:09.690201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c077db-3d2d-409e-9f2a-be9a2cf95bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:09.691094 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c077db-3d2d-409e-9f2a-be9a2cf95bc9 None admin] PATH_INFO: `/v3/projects/647004410b244685b36a7d774ed53f18` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:09.692259 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c077db-3d2d-409e-9f2a-be9a2cf95bc9 None admin] RBAC: Authorizing `identity:get_project(project_id=647004410b244685b36a7d774ed53f18)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:09.693677 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c077db-3d2d-409e-9f2a-be9a2cf95bc9 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:09.695358 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 596/1191] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:17:09 2022] GET /identity/v3/projects/647004410b244685b36a7d774ed53f18 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:17:09.701194 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd35a4-eea0-42e5-ae21-2a56204da020 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:09.795604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd35a4-eea0-42e5-ae21-2a56204da020 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:09.828799 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac0acc-2412-4047-ada6-9f67dfd1e5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:09.829666 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac0acc-2412-4047-ada6-9f67dfd1e5ef None admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:09.829945 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac0acc-2412-4047-ada6-9f67dfd1e5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:09.830219 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac0acc-2412-4047-ada6-9f67dfd1e5ef None admin] PATH_INFO: `/v3/projects/647004410b244685b36a7d774ed53f18/users/01a15cf338a64e579a964fbb770410fe/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:09.832210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ac0acc-2412-4047-ada6-9f67dfd1e5ef None admin] RBAC: Authorizing `identity:create_grant(project_id=647004410b244685b36a7d774ed53f18, user_id=01a15cf338a64e579a964fbb770410fe, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:09.833615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ac0acc-2412-4047-ada6-9f67dfd1e5ef None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:09.900710 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 596/1192] 10.222.0.24 () {58 vars in 1439 bytes} [Fri May 27 03:17:09 2022] PUT /identity/v3/projects/647004410b244685b36a7d774ed53f18/users/01a15cf338a64e579a964fbb770410fe/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:17:12.057255 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa371a-c35f-4c31-89d5-22eb9735fd68 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:12.057819 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa371a-c35f-4c31-89d5-22eb9735fd68 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:12.057819 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa371a-c35f-4c31-89d5-22eb9735fd68 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:12.059323 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 597/1193] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:12.064400 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea035a30-84c3-4da3-b156-13fd5bd61524 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:12.064803 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea035a30-84c3-4da3-b156-13fd5bd61524 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:12.064974 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea035a30-84c3-4da3-b156-13fd5bd61524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:12.082202 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-ea035a30-84c3-4da3-b156-13fd5bd61524 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:12.083416 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ea035a30-84c3-4da3-b156-13fd5bd61524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:12.103787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:17:12.149502 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 597/1194] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:17:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:17:12.172616 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa371a-c35f-4c31-89d5-22eb9735fd68 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:12.175504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:17:12.270980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa371a-c35f-4c31-89d5-22eb9735fd68 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:12.302415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08ba6a-9da8-4b89-bb6d-15d5346d09be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:12.303212 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08ba6a-9da8-4b89-bb6d-15d5346d09be None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:12.303492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08ba6a-9da8-4b89-bb6d-15d5346d09be None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:12.303762 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08ba6a-9da8-4b89-bb6d-15d5346d09be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:12.313453 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e08ba6a-9da8-4b89-bb6d-15d5346d09be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:17:12.321432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-5e08ba6a-9da8-4b89-bb6d-15d5346d09be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:17:12.321432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:17:12.321432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:17:12.321432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:17:12.321432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:17:12.321432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:17:12.321432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:17:12.321432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:17:12.321432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:17:12.321432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:17:12.321432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:17:12.321432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:17:12.321432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:17:12.321432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:17:12.321432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:17:12.321432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:17:12.322503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:17:12.322503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:17:12.322503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:17:12.322503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:17:12.322503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:17:12.322503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:17:12.322503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:17:12.322503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:17:12.322503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:17:12.322503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:17:12.322503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:17:12.322503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:17:12.322503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:17:12.322503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:17:12.322503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:17:12.322503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:17:12.323550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:17:12.323550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:17:12.323550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:17:12.323550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:17:12.323550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:17:12.323550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:17:12.323550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:17:12.323550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:17:12.323550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:17:12.323550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:17:12.323550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:17:12.323550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:17:12.323550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 598/1195] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:17:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:17:12.328832 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea035a30-84c3-4da3-b156-13fd5bd61524 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:12.439353 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea035a30-84c3-4da3-b156-13fd5bd61524 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:12.472436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d72d8a-0832-4dec-8185-c7c4c63ce1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:12.473096 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d72d8a-0832-4dec-8185-c7c4c63ce1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:12.473378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d72d8a-0832-4dec-8185-c7c4c63ce1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:12.473649 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d72d8a-0832-4dec-8185-c7c4c63ce1b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:12.474386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d72d8a-0832-4dec-8185-c7c4c63ce1b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:17:12.474691 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d72d8a-0832-4dec-8185-c7c4c63ce1b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:12.476076 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d72d8a-0832-4dec-8185-c7c4c63ce1b2 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:12.483833 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 598/1196] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:17:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:17:12.489592 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08ba6a-9da8-4b89-bb6d-15d5346d09be None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:12.548395 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:17:12.585437 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08ba6a-9da8-4b89-bb6d-15d5346d09be None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:12.617462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de1fd6-e775-4821-b09a-205b1f499a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:12.618312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de1fd6-e775-4821-b09a-205b1f499a98 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:12.618590 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de1fd6-e775-4821-b09a-205b1f499a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:12.618883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de1fd6-e775-4821-b09a-205b1f499a98 None admin] PATH_INFO: `/v3/users/01a15cf338a64e579a964fbb770410fe` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:12.620043 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27de1fd6-e775-4821-b09a-205b1f499a98 None admin] RBAC: Authorizing `identity:get_user(user_id=01a15cf338a64e579a964fbb770410fe)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:12.621455 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27de1fd6-e775-4821-b09a-205b1f499a98 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:12.623058 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 599/1197] 10.222.0.24 () {56 vars in 1177 bytes} [Fri May 27 03:17:12 2022] GET /identity/v3/users/01a15cf338a64e579a964fbb770410fe => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:17:12.628715 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d72d8a-0832-4dec-8185-c7c4c63ce1b2 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:12.647970 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:17:12.724828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d72d8a-0832-4dec-8185-c7c4c63ce1b2 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:12.756503 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea8138-c1bf-4142-b030-f2e3c82250f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:12.757132 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea8138-c1bf-4142-b030-f2e3c82250f8 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:12.757412 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea8138-c1bf-4142-b030-f2e3c82250f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:12.757682 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea8138-c1bf-4142-b030-f2e3c82250f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:12.821363 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea8138-c1bf-4142-b030-f2e3c82250f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:12.822792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea8138-c1bf-4142-b030-f2e3c82250f8 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:12.840006 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:17:12.916267 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 599/1198] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:17:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:17:12.922045 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de1fd6-e775-4821-b09a-205b1f499a98 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:13.017158 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de1fd6-e775-4821-b09a-205b1f499a98 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:13.043783 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:17:13.049145 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab0529-fda0-4d8f-a864-2ec3be1c0f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:13.050049 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab0529-fda0-4d8f-a864-2ec3be1c0f31 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:13.053313 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab0529-fda0-4d8f-a864-2ec3be1c0f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:13.053313 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab0529-fda0-4d8f-a864-2ec3be1c0f31 None admin] PATH_INFO: `/v3/projects/647004410b244685b36a7d774ed53f18` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:13.053313 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab0529-fda0-4d8f-a864-2ec3be1c0f31 None admin] RBAC: Authorizing `identity:get_project(project_id=647004410b244685b36a7d774ed53f18)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:13.053313 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab0529-fda0-4d8f-a864-2ec3be1c0f31 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:13.054844 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 600/1199] 10.222.0.24 () {56 vars in 1186 bytes} [Fri May 27 03:17:12 2022] GET /identity/v3/projects/647004410b244685b36a7d774ed53f18 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:17:13.060795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea8138-c1bf-4142-b030-f2e3c82250f8 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:13.156469 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea8138-c1bf-4142-b030-f2e3c82250f8 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:13.172090 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:17:13.188068 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d65113-18d3-4378-b5d5-28e650d17d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:13.188705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d65113-18d3-4378-b5d5-28e650d17d08 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:13.188983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d65113-18d3-4378-b5d5-28e650d17d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:13.189256 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d65113-18d3-4378-b5d5-28e650d17d08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:13.190591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d65113-18d3-4378-b5d5-28e650d17d08 None admin] RBAC: Adding query filter params (role.id=e5a908bd3ed5445b82a6053246533b88, scope.project.id=647004410b244685b36a7d774ed53f18, user.id=01a15cf338a64e579a964fbb770410fe) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:17:13.190939 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d65113-18d3-4378-b5d5-28e650d17d08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:13.192440 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d65113-18d3-4378-b5d5-28e650d17d08 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:13.198075 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 600/1200] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:17:13 2022] GET /identity/v3/role_assignments?user.id=01a15cf338a64e579a964fbb770410fe&scope.project.id=647004410b244685b36a7d774ed53f18&role.id=e5a908bd3ed5445b82a6053246533b88 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:17:23.078301 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:17:23.204074 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=9 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:31.971636 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08794540-242a-4b90-9049-9ec24dd3e448 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:31.972226 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08794540-242a-4b90-9049-9ec24dd3e448 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:31.972351 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08794540-242a-4b90-9049-9ec24dd3e448 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:31.973735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 601/1201] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:18:31.979062 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4837b237-60d2-48b1-8e33-0decb4c265dc None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:31.979511 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4837b237-60d2-48b1-8e33-0decb4c265dc None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:31.979640 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4837b237-60d2-48b1-8e33-0decb4c265dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:31.996413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-4837b237-60d2-48b1-8e33-0decb4c265dc None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:18:31.997581 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-4837b237-60d2-48b1-8e33-0decb4c265dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:18:32.063195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 601/1202] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:18:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:18:32.087692 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08794540-242a-4b90-9049-9ec24dd3e448 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:32.188615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08794540-242a-4b90-9049-9ec24dd3e448 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:32.221971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e758e4b-f926-40f6-ba43-c50308cfa6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:32.222693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e758e4b-f926-40f6-ba43-c50308cfa6de None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:32.223005 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e758e4b-f926-40f6-ba43-c50308cfa6de None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:32.223386 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e758e4b-f926-40f6-ba43-c50308cfa6de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:32.233625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e758e4b-f926-40f6-ba43-c50308cfa6de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:32.235088 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e758e4b-f926-40f6-ba43-c50308cfa6de None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:32.251875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-6e758e4b-f926-40f6-ba43-c50308cfa6de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:18:32.251875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:32.251875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:32.251875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:32.251875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:32.251875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:32.251875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:32.251875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:32.251875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:32.251875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:32.251875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:32.251875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:32.251875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:18:32.251875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:18:32.251875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:18:32.252981 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:18:32.252981 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:32.252981 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:32.252981 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:32.252981 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:32.252981 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:32.252981 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:32.252981 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:32.252981 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:18:32.252981 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:32.252981 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:32.252981 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:32.252981 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:32.252981 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:32.252981 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:32.252981 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:32.252981 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:32.254131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:18:32.254131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:18:32.254131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:18:32.254131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:18:32.254131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 602/1203] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:18:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:32.259104 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4837b237-60d2-48b1-8e33-0decb4c265dc None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:32.363217 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4837b237-60d2-48b1-8e33-0decb4c265dc None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:32.391836 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:32.397524 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a299b2df-bd92-4ae3-b6fe-59759e09ae48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:32.398184 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299b2df-bd92-4ae3-b6fe-59759e09ae48 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:32.398468 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299b2df-bd92-4ae3-b6fe-59759e09ae48 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:32.398747 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a299b2df-bd92-4ae3-b6fe-59759e09ae48 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:32.399498 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a299b2df-bd92-4ae3-b6fe-59759e09ae48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:32.399806 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a299b2df-bd92-4ae3-b6fe-59759e09ae48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:32.401167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a299b2df-bd92-4ae3-b6fe-59759e09ae48 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:32.412249 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 602/1204] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:32.417794 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e758e4b-f926-40f6-ba43-c50308cfa6de None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:32.470527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:32.516853 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e758e4b-f926-40f6-ba43-c50308cfa6de None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:32.549338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3632a4a-3662-47ec-b491-8b9e02ae4cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:32.550001 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3632a4a-3662-47ec-b491-8b9e02ae4cc1 None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:32.550283 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3632a4a-3662-47ec-b491-8b9e02ae4cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:32.550558 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3632a4a-3662-47ec-b491-8b9e02ae4cc1 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:32.551300 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3632a4a-3662-47ec-b491-8b9e02ae4cc1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:32.552744 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3632a4a-3662-47ec-b491-8b9e02ae4cc1 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:32.642135 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 603/1205] 10.222.0.24 () {60 vars in 1132 bytes} [Fri May 27 03:18:32 2022] POST /identity/v3/users => generated 251 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:18:34.831315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a839e-f49e-4981-a56d-7e2484604e8a None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:34.831866 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a839e-f49e-4981-a56d-7e2484604e8a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:34.831866 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a839e-f49e-4981-a56d-7e2484604e8a None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:34.833285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 603/1206] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:18:34.838770 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175959c-27d3-447b-963f-0b340c1d99c8 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:34.839177 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175959c-27d3-447b-963f-0b340c1d99c8 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:34.839359 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175959c-27d3-447b-963f-0b340c1d99c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:34.856299 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-5175959c-27d3-447b-963f-0b340c1d99c8 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:18:34.857508 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5175959c-27d3-447b-963f-0b340c1d99c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:18:34.922612 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 604/1207] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:18:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:18:34.946156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a839e-f49e-4981-a56d-7e2484604e8a None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:35.046631 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a839e-f49e-4981-a56d-7e2484604e8a None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:35.080838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9fc1a4-0e82-4a5f-8773-d0757c183168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:35.081666 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9fc1a4-0e82-4a5f-8773-d0757c183168 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:35.081899 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9fc1a4-0e82-4a5f-8773-d0757c183168 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:35.082163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9fc1a4-0e82-4a5f-8773-d0757c183168 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:35.092258 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9fc1a4-0e82-4a5f-8773-d0757c183168 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:18:35.107201 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-bb9fc1a4-0e82-4a5f-8773-d0757c183168 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:18:35.107201 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:35.107201 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:35.107201 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:35.107201 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:35.107201 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:35.107201 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:35.107201 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:35.107201 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:35.107201 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:35.107201 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:35.107201 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:35.107201 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:18:35.107201 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:18:35.107201 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:18:35.107201 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:18:35.108369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:18:35.108369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:18:35.108369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:18:35.108369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:18:35.108369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:35.108369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:35.108369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:35.108369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:35.108369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:35.108369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:35.108369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:35.108369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:18:35.108369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:35.108369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:35.108369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:35.108369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:35.109394 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:35.109394 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:35.109394 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:35.109394 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:35.109394 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:18:35.109394 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:18:35.109394 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:18:35.109394 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:18:35.109394 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:18:35.109394 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:18:35.109394 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:18:35.109394 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:18:35.109394 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 604/1208] 10.222.0.24 () {56 vars in 1102 bytes} [Fri May 27 03:18:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:35.115003 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175959c-27d3-447b-963f-0b340c1d99c8 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:35.156385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:35.213887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175959c-27d3-447b-963f-0b340c1d99c8 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:35.247066 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32298e-f04b-4e1a-b503-a1ceb9d8460f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:35.247727 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32298e-f04b-4e1a-b503-a1ceb9d8460f None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:35.248007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32298e-f04b-4e1a-b503-a1ceb9d8460f None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:35.248280 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32298e-f04b-4e1a-b503-a1ceb9d8460f None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:35.249015 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a32298e-f04b-4e1a-b503-a1ceb9d8460f None admin] RBAC: Adding query filter params (name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:35.249321 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a32298e-f04b-4e1a-b503-a1ceb9d8460f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:35.250759 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a32298e-f04b-4e1a-b503-a1ceb9d8460f None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:35.258425 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 605/1209] 10.222.0.24 () {56 vars in 1103 bytes} [Fri May 27 03:18:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:35.264156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9fc1a4-0e82-4a5f-8773-d0757c183168 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:35.313571 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:35.362925 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9fc1a4-0e82-4a5f-8773-d0757c183168 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:35.395346 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b03c0-bbe6-44f4-9fd4-f3f87eb1ea3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:35.396119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b03c0-bbe6-44f4-9fd4-f3f87eb1ea3e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:35.396400 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b03c0-bbe6-44f4-9fd4-f3f87eb1ea3e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:35.396675 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b03c0-bbe6-44f4-9fd4-f3f87eb1ea3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:35.406864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b03c0-bbe6-44f4-9fd4-f3f87eb1ea3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:35.408314 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b03c0-bbe6-44f4-9fd4-f3f87eb1ea3e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:35.425809 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-231b03c0-bbe6-44f4-9fd4-f3f87eb1ea3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:18:35.425809 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:35.425809 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:35.425809 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:35.425809 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:35.425809 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:35.425809 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:35.425809 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:35.425809 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:35.425809 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:35.425809 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:35.425809 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:35.425809 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:18:35.425809 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:18:35.425809 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:18:35.432879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:18:35.432879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:35.432879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:35.432879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:35.432879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:35.432879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:35.432879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:35.432879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:35.432879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:18:35.432879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:35.432879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:35.432879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:35.432879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:35.432879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:35.432879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:35.432879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:35.432879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:35.433917 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:18:35.433917 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:18:35.433917 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:18:35.433917 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:18:35.433917 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 605/1210] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:18:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:35.434250 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32298e-f04b-4e1a-b503-a1ceb9d8460f None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:35.528897 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:35.531720 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32298e-f04b-4e1a-b503-a1ceb9d8460f None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:35.563850 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53be643-bd55-4617-8892-42e1b9a9580b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:35.564829 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53be643-bd55-4617-8892-42e1b9a9580b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:35.564829 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53be643-bd55-4617-8892-42e1b9a9580b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:35.565059 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53be643-bd55-4617-8892-42e1b9a9580b None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:35.565768 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53be643-bd55-4617-8892-42e1b9a9580b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:35.566073 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53be643-bd55-4617-8892-42e1b9a9580b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:35.567457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53be643-bd55-4617-8892-42e1b9a9580b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:35.579418 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 606/1211] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:18:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:35.585052 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b03c0-bbe6-44f4-9fd4-f3f87eb1ea3e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:35.681415 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b03c0-bbe6-44f4-9fd4-f3f87eb1ea3e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:35.682829 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:35.715635 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417a387-876f-4c49-ae0b-d1cfc594d839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:35.716441 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417a387-876f-4c49-ae0b-d1cfc594d839 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:35.716721 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417a387-876f-4c49-ae0b-d1cfc594d839 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:35.716993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9417a387-876f-4c49-ae0b-d1cfc594d839 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:35.724098 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9417a387-876f-4c49-ae0b-d1cfc594d839 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:35.725581 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9417a387-876f-4c49-ae0b-d1cfc594d839 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:35.741605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-9417a387-876f-4c49-ae0b-d1cfc594d839 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 03:18:35.741605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:35.741605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:35.741605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:35.741605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:35.741605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:35.741605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:35.741605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:35.741605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:35.741605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:35.741605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:35.741605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:35.741605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:18:35.741605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:18:35.741605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:18:35.741605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:18:35.742706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:35.742706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:35.742706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:18:35.742706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:18:35.742706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:18:35.742706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:18:35.742706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:35.742706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:35.742706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:35.742706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:35.742706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:35.742706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:18:35.742706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:35.742706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:35.742706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:35.742706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:35.742706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:35.742706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:35.743973 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:35.743973 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:35.743973 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:18:35.743973 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:18:35.743973 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:18:35.743973 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:18:35.743973 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:18:35.743973 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:18:35.743973 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 03:18:35.743973 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:18:35.743973 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 606/1212] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:18:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:35.748775 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53be643-bd55-4617-8892-42e1b9a9580b None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:35.846553 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53be643-bd55-4617-8892-42e1b9a9580b None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:35.879267 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3510957-7d18-4b63-9833-d0fd6eabc6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:35.879917 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3510957-7d18-4b63-9833-d0fd6eabc6bb None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:35.880193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3510957-7d18-4b63-9833-d0fd6eabc6bb None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:35.880461 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3510957-7d18-4b63-9833-d0fd6eabc6bb None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:35.881280 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3510957-7d18-4b63-9833-d0fd6eabc6bb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:35.881585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3510957-7d18-4b63-9833-d0fd6eabc6bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:35.883018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3510957-7d18-4b63-9833-d0fd6eabc6bb None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:35.899523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 607/1213] 10.222.0.24 () {56 vars in 1143 bytes} [Fri May 27 03:18:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:35.905358 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417a387-876f-4c49-ae0b-d1cfc594d839 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:36.002682 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9417a387-876f-4c49-ae0b-d1cfc594d839 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:36.036202 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d539d7-5c1c-44fb-8484-bfba585857ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:36.036833 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d539d7-5c1c-44fb-8484-bfba585857ba None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:36.037113 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d539d7-5c1c-44fb-8484-bfba585857ba None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:36.037389 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d539d7-5c1c-44fb-8484-bfba585857ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:36.065382 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:36.102586 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d539d7-5c1c-44fb-8484-bfba585857ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:36.104076 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d539d7-5c1c-44fb-8484-bfba585857ba None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:36.199089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 607/1214] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:18:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:18:36.205395 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3510957-7d18-4b63-9833-d0fd6eabc6bb None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:36.223796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:36.300652 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3510957-7d18-4b63-9833-d0fd6eabc6bb None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:36.333617 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca4004f-5b70-4aa1-95de-9ad761015224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:36.334393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4004f-5b70-4aa1-95de-9ad761015224 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:36.334673 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4004f-5b70-4aa1-95de-9ad761015224 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:36.334977 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4004f-5b70-4aa1-95de-9ad761015224 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:36.344737 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca4004f-5b70-4aa1-95de-9ad761015224 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:36.346151 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca4004f-5b70-4aa1-95de-9ad761015224 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:36.366419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-aca4004f-5b70-4aa1-95de-9ad761015224 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:18:36.366419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:36.366419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:36.366419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:36.366419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:36.366419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:36.366419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:36.366419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:36.366419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:36.366419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:36.366419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:36.366419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:36.366419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:18:36.366419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:18:36.366419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:18:36.378303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:18:36.378303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:36.378303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:36.378303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:36.378303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:36.378303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:36.378303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:36.378303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:36.378303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:18:36.378303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:36.378303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:36.378303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:36.378303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:36.378303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:36.378303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:36.378303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:36.378303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:36.379265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:18:36.379265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:18:36.379265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:18:36.379265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:18:36.379265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 608/1215] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:18:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:36.379619 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d539d7-5c1c-44fb-8484-bfba585857ba None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:36.431988 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:36.477537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d539d7-5c1c-44fb-8484-bfba585857ba None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:36.513980 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49670e-2af5-433d-83a0-72a6fdea5b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:36.514675 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49670e-2af5-433d-83a0-72a6fdea5b17 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:36.515013 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49670e-2af5-433d-83a0-72a6fdea5b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:36.515290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f49670e-2af5-433d-83a0-72a6fdea5b17 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:36.515999 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f49670e-2af5-433d-83a0-72a6fdea5b17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:36.516300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f49670e-2af5-433d-83a0-72a6fdea5b17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:36.517643 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f49670e-2af5-433d-83a0-72a6fdea5b17 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:36.528693 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 608/1216] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:18:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:36.534442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca4004f-5b70-4aa1-95de-9ad761015224 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:36.601268 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:36.630365 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca4004f-5b70-4aa1-95de-9ad761015224 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:36.662789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea554be0-117e-400a-a33d-3cc41a07d89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:36.663585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea554be0-117e-400a-a33d-3cc41a07d89b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:36.663865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea554be0-117e-400a-a33d-3cc41a07d89b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:36.664135 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea554be0-117e-400a-a33d-3cc41a07d89b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:36.671098 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea554be0-117e-400a-a33d-3cc41a07d89b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:36.672585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea554be0-117e-400a-a33d-3cc41a07d89b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:36.685918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-ea554be0-117e-400a-a33d-3cc41a07d89b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:18:36.685918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:36.685918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:36.685918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:36.685918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:36.685918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:36.685918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:36.685918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:36.685918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:36.685918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:36.685918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:36.685918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:36.685918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:18:36.685918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:18:36.685918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:18:36.687881 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:18:36.687881 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:36.687881 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:36.687881 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:36.687881 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:36.687881 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:36.687881 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:36.687881 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:36.687881 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:18:36.687881 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:36.687881 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:36.687881 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:36.687881 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:36.687881 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:36.687881 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:36.687881 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:36.687881 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:36.689126 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:18:36.689126 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:18:36.689126 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:18:36.689126 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:18:36.689126 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:18:36.689126 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:18:36.689126 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:18:36.689126 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:18:36.689126 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 609/1217] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:18:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:18:36.693473 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49670e-2af5-433d-83a0-72a6fdea5b17 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:36.809678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f49670e-2af5-433d-83a0-72a6fdea5b17 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:36.831913 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:36.843448 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258941d6-a046-4777-a342-a1586469bee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:36.844103 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258941d6-a046-4777-a342-a1586469bee2 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:36.844386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258941d6-a046-4777-a342-a1586469bee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:36.844663 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258941d6-a046-4777-a342-a1586469bee2 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:36.845450 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258941d6-a046-4777-a342-a1586469bee2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:36.845761 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258941d6-a046-4777-a342-a1586469bee2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:36.847242 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258941d6-a046-4777-a342-a1586469bee2 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:36.858803 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 609/1218] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:18:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:36.864913 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea554be0-117e-400a-a33d-3cc41a07d89b None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:36.962509 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea554be0-117e-400a-a33d-3cc41a07d89b None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:36.989988 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:36.995703 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbc48e-312b-4539-a6ed-1a646e8282bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:36.996348 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbc48e-312b-4539-a6ed-1a646e8282bd None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:36.996634 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbc48e-312b-4539-a6ed-1a646e8282bd None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:36.996912 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbc48e-312b-4539-a6ed-1a646e8282bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:36.997731 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbc48e-312b-4539-a6ed-1a646e8282bd None admin] RBAC: Adding query filter params (role.id=8e315ffd561a48f0891ef53b51d548d8, scope.project.id=6ceffdd955324aec84ba048d2137ae00, user.id=95f2d0aceef4451a83ee89fac5130a0c) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:36.998045 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbc48e-312b-4539-a6ed-1a646e8282bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:36.999986 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbc48e-312b-4539-a6ed-1a646e8282bd None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:37.005232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 610/1219] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:18:36 2022] GET /identity/v3/role_assignments?user.id=95f2d0aceef4451a83ee89fac5130a0c&scope.project.id=6ceffdd955324aec84ba048d2137ae00&role.id=8e315ffd561a48f0891ef53b51d548d8 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:39.179566 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df784a-646f-467b-b123-71b64459c5ed None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:39.180105 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df784a-646f-467b-b123-71b64459c5ed None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:39.180105 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30df784a-646f-467b-b123-71b64459c5ed None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:39.181495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 610/1220] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:18:39.187504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000aa69-cd43-47db-891e-a28309d88a25 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:39.187944 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000aa69-cd43-47db-891e-a28309d88a25 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:39.188056 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000aa69-cd43-47db-891e-a28309d88a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:39.205321 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-8000aa69-cd43-47db-891e-a28309d88a25 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:18:39.206947 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8000aa69-cd43-47db-891e-a28309d88a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:18:39.275496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 611/1221] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:18:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:18:39.297987 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df784a-646f-467b-b123-71b64459c5ed None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:39.399673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30df784a-646f-467b-b123-71b64459c5ed None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:39.431930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313de8f4-0be1-4edb-9368-cee87e3af980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:39.432712 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313de8f4-0be1-4edb-9368-cee87e3af980 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:39.432991 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313de8f4-0be1-4edb-9368-cee87e3af980 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:39.434649 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313de8f4-0be1-4edb-9368-cee87e3af980 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:39.443550 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313de8f4-0be1-4edb-9368-cee87e3af980 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:18:39.451409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-313de8f4-0be1-4edb-9368-cee87e3af980 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:18:39.451409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:39.451409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:39.451409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:39.451409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:39.451409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:39.451409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:39.451409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:39.451409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:39.451409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:39.451409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:39.451409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:39.451409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:18:39.451409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:18:39.451409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:18:39.451409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:18:39.452479 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:18:39.452479 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:18:39.452479 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:18:39.452479 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:18:39.452479 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:39.452479 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:39.452479 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:39.452479 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:39.452479 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:39.452479 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:39.452479 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:39.452479 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:18:39.452479 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:39.452479 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:39.452479 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:39.452479 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:39.453317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:39.453317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:39.453317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:39.453317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:39.453317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:18:39.453317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:18:39.453317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:18:39.453317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:18:39.453317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:18:39.453317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:18:39.453317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:18:39.453317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:18:39.453317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 611/1222] 10.222.0.24 () {56 vars in 1102 bytes} [Fri May 27 03:18:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:39.458606 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8000aa69-cd43-47db-891e-a28309d88a25 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:39.560939 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8000aa69-cd43-47db-891e-a28309d88a25 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:39.585174 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:39.595580 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e6907-360d-4ac6-a5be-2d2fd2ecb7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:39.596267 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e6907-360d-4ac6-a5be-2d2fd2ecb7bb None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:39.596547 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e6907-360d-4ac6-a5be-2d2fd2ecb7bb None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:39.596818 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e6907-360d-4ac6-a5be-2d2fd2ecb7bb None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:39.597554 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e6907-360d-4ac6-a5be-2d2fd2ecb7bb None admin] RBAC: Adding query filter params (name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:39.597856 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e6907-360d-4ac6-a5be-2d2fd2ecb7bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:39.599255 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e6907-360d-4ac6-a5be-2d2fd2ecb7bb None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:39.606830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 612/1223] 10.222.0.24 () {56 vars in 1103 bytes} [Fri May 27 03:18:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:39.613138 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313de8f4-0be1-4edb-9368-cee87e3af980 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:39.655968 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:39.711695 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313de8f4-0be1-4edb-9368-cee87e3af980 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:39.745229 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c1b0d-28f4-4b4d-9a27-61e8e1b2d43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:39.746014 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c1b0d-28f4-4b4d-9a27-61e8e1b2d43d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:39.746306 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c1b0d-28f4-4b4d-9a27-61e8e1b2d43d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:39.746581 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c1b0d-28f4-4b4d-9a27-61e8e1b2d43d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:39.757556 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c1b0d-28f4-4b4d-9a27-61e8e1b2d43d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:39.759055 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6c1b0d-28f4-4b4d-9a27-61e8e1b2d43d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:39.775624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-2f6c1b0d-28f4-4b4d-9a27-61e8e1b2d43d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:18:39.775624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:39.775624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:39.775624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:39.775624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:39.775624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:39.775624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:39.775624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:39.775624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:39.775624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:39.775624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:39.775624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:39.775624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:18:39.775624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:18:39.775624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:18:39.776700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:18:39.776700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:39.776700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:39.776700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:39.776700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:39.776700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:39.776700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:39.776700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:39.776700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:18:39.776700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:39.776700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:39.776700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:39.776700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:39.776700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:39.776700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:39.776700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:39.776700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:39.777642 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:18:39.777642 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:18:39.777642 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:18:39.777642 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:18:39.777642 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 612/1224] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:18:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:18:39.782536 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e6907-360d-4ac6-a5be-2d2fd2ecb7bb None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:39.882558 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e6907-360d-4ac6-a5be-2d2fd2ecb7bb None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:39.915115 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff9ec0-8b7a-4067-91a8-ba609aa81f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:39.915709 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff9ec0-8b7a-4067-91a8-ba609aa81f95 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:39.915991 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff9ec0-8b7a-4067-91a8-ba609aa81f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:39.916264 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff9ec0-8b7a-4067-91a8-ba609aa81f95 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:39.916976 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff9ec0-8b7a-4067-91a8-ba609aa81f95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:39.917280 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff9ec0-8b7a-4067-91a8-ba609aa81f95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:39.918667 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff9ec0-8b7a-4067-91a8-ba609aa81f95 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:39.930637 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 613/1225] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:39.936869 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c1b0d-28f4-4b4d-9a27-61e8e1b2d43d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:40.026957 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:40.033845 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c1b0d-28f4-4b4d-9a27-61e8e1b2d43d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:40.065842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b65c4-d403-4264-b3f2-92936b7fa0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:40.066623 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b65c4-d403-4264-b3f2-92936b7fa0d8 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:40.066924 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b65c4-d403-4264-b3f2-92936b7fa0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:40.067193 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b65c4-d403-4264-b3f2-92936b7fa0d8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:40.074156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b65c4-d403-4264-b3f2-92936b7fa0d8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:40.075689 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b65c4-d403-4264-b3f2-92936b7fa0d8 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:40.090395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-d24b65c4-d403-4264-b3f2-92936b7fa0d8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 03:18:40.090395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:40.090395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:40.090395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:40.090395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:40.090395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:40.090395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:40.090395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:40.090395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:40.090395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:40.090395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:40.090395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:40.090395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:18:40.090395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:18:40.090395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:18:40.090395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:18:40.091590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:40.091590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:40.091590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:18:40.091590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:18:40.091590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:18:40.091590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:18:40.091590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:40.091590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:40.091590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:40.091590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:40.091590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:40.091590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:18:40.091590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:40.091590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:40.091590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:40.091590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:40.091590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:40.091590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:40.093109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:40.093109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:40.093109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:18:40.093109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:18:40.093109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:18:40.093109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:18:40.093109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:18:40.093109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:18:40.093109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 03:18:40.093109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:18:40.093109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 613/1226] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:18:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:40.097743 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff9ec0-8b7a-4067-91a8-ba609aa81f95 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:40.144254 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:40.195321 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff9ec0-8b7a-4067-91a8-ba609aa81f95 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:40.229563 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6519b9-5cc3-478a-9a03-bd75f8d1b342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:40.230221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6519b9-5cc3-478a-9a03-bd75f8d1b342 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:40.230499 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6519b9-5cc3-478a-9a03-bd75f8d1b342 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:40.230768 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6519b9-5cc3-478a-9a03-bd75f8d1b342 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:40.231619 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6519b9-5cc3-478a-9a03-bd75f8d1b342 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:40.231925 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6519b9-5cc3-478a-9a03-bd75f8d1b342 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:40.233835 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6519b9-5cc3-478a-9a03-bd75f8d1b342 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:40.250817 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 614/1227] 10.222.0.24 () {56 vars in 1143 bytes} [Fri May 27 03:18:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:40.256433 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b65c4-d403-4264-b3f2-92936b7fa0d8 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:40.352662 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b65c4-d403-4264-b3f2-92936b7fa0d8 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:40.393232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab44e5-075b-4256-a1d6-9f98fd9b4195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:40.394020 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab44e5-075b-4256-a1d6-9f98fd9b4195 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:40.394299 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab44e5-075b-4256-a1d6-9f98fd9b4195 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:40.394571 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab44e5-075b-4256-a1d6-9f98fd9b4195 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:40.404403 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dab44e5-075b-4256-a1d6-9f98fd9b4195 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:40.405911 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dab44e5-075b-4256-a1d6-9f98fd9b4195 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:40.423615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-8dab44e5-075b-4256-a1d6-9f98fd9b4195 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:18:40.423615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:40.423615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:40.423615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:40.423615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:40.423615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:40.423615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:40.423615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:40.423615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:40.423615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:40.423615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:40.423615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:40.423615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:18:40.423615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:18:40.423615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:18:40.424672 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:18:40.424672 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:40.424672 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:40.424672 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:40.424672 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:40.424672 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:40.424672 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:40.424672 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:40.424672 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:18:40.424672 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:40.424672 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:40.424672 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:40.424672 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:40.424672 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:40.424672 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:40.424672 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:40.424672 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:40.425690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:18:40.425690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:18:40.425690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:18:40.425690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:18:40.425690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 614/1228] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:18:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:40.430411 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6519b9-5cc3-478a-9a03-bd75f8d1b342 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:40.536651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6519b9-5cc3-478a-9a03-bd75f8d1b342 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:40.539553 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:40.571540 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be056e7-16b7-4745-98ba-9b89ade5a9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:40.572200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be056e7-16b7-4745-98ba-9b89ade5a9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:40.572482 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be056e7-16b7-4745-98ba-9b89ade5a9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:40.572756 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be056e7-16b7-4745-98ba-9b89ade5a9a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:40.573473 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be056e7-16b7-4745-98ba-9b89ade5a9a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:40.573778 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be056e7-16b7-4745-98ba-9b89ade5a9a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:40.575417 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be056e7-16b7-4745-98ba-9b89ade5a9a9 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:40.586821 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 615/1229] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:40.593226 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab44e5-075b-4256-a1d6-9f98fd9b4195 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:40.602152 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:40.690311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab44e5-075b-4256-a1d6-9f98fd9b4195 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:40.722199 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8906eafb-4c42-46e7-9067-d19300d3bb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:40.723078 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906eafb-4c42-46e7-9067-d19300d3bb31 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:40.723360 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906eafb-4c42-46e7-9067-d19300d3bb31 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:40.723633 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906eafb-4c42-46e7-9067-d19300d3bb31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:40.730041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8906eafb-4c42-46e7-9067-d19300d3bb31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:40.731500 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8906eafb-4c42-46e7-9067-d19300d3bb31 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:40.745886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-8906eafb-4c42-46e7-9067-d19300d3bb31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:18:40.745886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:40.745886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:40.745886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:40.745886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:40.745886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:40.745886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:40.745886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:40.745886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:40.745886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:40.745886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:40.745886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:40.745886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:18:40.745886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:18:40.745886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:18:40.749986 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:18:40.749986 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:40.749986 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:40.749986 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:40.749986 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:40.749986 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:40.749986 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:40.749986 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:40.749986 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:18:40.749986 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:40.749986 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:40.749986 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:40.749986 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:40.749986 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:40.749986 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:40.749986 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:40.749986 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:40.751163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:18:40.751163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:18:40.751163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:18:40.751163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:18:40.751163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:18:40.751163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:18:40.751163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:18:40.751163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:18:40.751163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 615/1230] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:18:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:40.753465 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be056e7-16b7-4745-98ba-9b89ade5a9a9 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:40.855972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be056e7-16b7-4745-98ba-9b89ade5a9a9 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:40.894737 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff916b75-2dec-4a19-9468-32328b3b0fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:40.895422 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff916b75-2dec-4a19-9468-32328b3b0fcb None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:40.895706 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff916b75-2dec-4a19-9468-32328b3b0fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:40.895984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff916b75-2dec-4a19-9468-32328b3b0fcb None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:40.896774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff916b75-2dec-4a19-9468-32328b3b0fcb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:40.897084 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff916b75-2dec-4a19-9468-32328b3b0fcb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:40.898503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff916b75-2dec-4a19-9468-32328b3b0fcb None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:40.910738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 616/1231] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:18:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:40.916958 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8906eafb-4c42-46e7-9067-d19300d3bb31 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:40.984669 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:41.018477 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8906eafb-4c42-46e7-9067-d19300d3bb31 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:41.051719 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c787c74-073b-4421-9e5f-9484f67bae72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:41.052684 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c787c74-073b-4421-9e5f-9484f67bae72 None admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:41.052962 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c787c74-073b-4421-9e5f-9484f67bae72 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:41.053229 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c787c74-073b-4421-9e5f-9484f67bae72 None admin] PATH_INFO: `/v3/projects/6ceffdd955324aec84ba048d2137ae00/users/95f2d0aceef4451a83ee89fac5130a0c/roles/8e315ffd561a48f0891ef53b51d548d8` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:41.067693 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c787c74-073b-4421-9e5f-9484f67bae72 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ceffdd955324aec84ba048d2137ae00, user_id=95f2d0aceef4451a83ee89fac5130a0c, role_id=8e315ffd561a48f0891ef53b51d548d8)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:41.069154 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c787c74-073b-4421-9e5f-9484f67bae72 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:41.160230 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 616/1232] 10.222.0.24 () {58 vars in 1439 bytes} [Fri May 27 03:18:40 2022] PUT /identity/v3/projects/6ceffdd955324aec84ba048d2137ae00/users/95f2d0aceef4451a83ee89fac5130a0c/roles/8e315ffd561a48f0891ef53b51d548d8 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:18:43.322739 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eee156-be71-44b6-8ad2-5dad0a5e3d57 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:43.323286 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eee156-be71-44b6-8ad2-5dad0a5e3d57 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:43.323402 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eee156-be71-44b6-8ad2-5dad0a5e3d57 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:43.324973 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 617/1233] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:18:43.330759 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c394bb5-00f5-49cf-9403-e58a421c2c03 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:43.331215 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c394bb5-00f5-49cf-9403-e58a421c2c03 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:43.331308 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c394bb5-00f5-49cf-9403-e58a421c2c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:43.348218 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-4c394bb5-00f5-49cf-9403-e58a421c2c03 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:18:43.349394 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-4c394bb5-00f5-49cf-9403-e58a421c2c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:18:43.413043 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 617/1234] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:18:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:18:43.436868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eee156-be71-44b6-8ad2-5dad0a5e3d57 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:43.461637 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:43.537831 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eee156-be71-44b6-8ad2-5dad0a5e3d57 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:43.572019 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f423f-383c-4914-b965-09e51c543610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:43.572807 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f423f-383c-4914-b965-09e51c543610 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:43.573088 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f423f-383c-4914-b965-09e51c543610 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:43.573361 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f423f-383c-4914-b965-09e51c543610 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:43.596987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724f423f-383c-4914-b965-09e51c543610 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:18:43.596987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-724f423f-383c-4914-b965-09e51c543610 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:18:43.596987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:43.596987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:43.596987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:43.596987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:43.596987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:43.596987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:43.596987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:43.596987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:43.596987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:43.596987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:43.596987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:43.596987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:18:43.597952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:18:43.597952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:18:43.597952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:18:43.597952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:18:43.597952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:18:43.597952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:18:43.597952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:18:43.597952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:43.597952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:43.597952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:43.597952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:43.597952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:43.597952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:43.597952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:43.597952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:18:43.597952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:43.597952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:43.598875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:43.598875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:43.598875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:43.598875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:43.598875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:43.598875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:43.598875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:18:43.598875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:18:43.598875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:18:43.598875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:18:43.598875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:18:43.598875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:18:43.598875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:18:43.598875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:18:43.598875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 618/1235] 10.222.0.24 () {56 vars in 1102 bytes} [Fri May 27 03:18:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:43.599840 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c394bb5-00f5-49cf-9403-e58a421c2c03 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:43.643078 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:43.693313 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c394bb5-00f5-49cf-9403-e58a421c2c03 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:43.725280 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c74c2f-8b7e-4d88-a3aa-e27e09b206fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:43.725927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c74c2f-8b7e-4d88-a3aa-e27e09b206fb None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:43.726206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c74c2f-8b7e-4d88-a3aa-e27e09b206fb None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:43.726479 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c74c2f-8b7e-4d88-a3aa-e27e09b206fb None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:43.727241 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c74c2f-8b7e-4d88-a3aa-e27e09b206fb None admin] RBAC: Adding query filter params (name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:43.727546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c74c2f-8b7e-4d88-a3aa-e27e09b206fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:43.728897 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c74c2f-8b7e-4d88-a3aa-e27e09b206fb None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:43.737752 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 618/1236] 10.222.0.24 () {56 vars in 1103 bytes} [Fri May 27 03:18:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:43.744060 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f423f-383c-4914-b965-09e51c543610 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:43.825020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:43.839989 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f423f-383c-4914-b965-09e51c543610 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:43.873928 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d499b0b-84ae-450d-b449-fc11a613d01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:43.874634 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d499b0b-84ae-450d-b449-fc11a613d01b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:43.874940 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d499b0b-84ae-450d-b449-fc11a613d01b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:43.875212 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d499b0b-84ae-450d-b449-fc11a613d01b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:43.884945 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d499b0b-84ae-450d-b449-fc11a613d01b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:43.886393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d499b0b-84ae-450d-b449-fc11a613d01b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:43.903347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-9d499b0b-84ae-450d-b449-fc11a613d01b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:18:43.903347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:43.903347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:43.903347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:43.903347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:43.903347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:43.903347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:43.903347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:43.903347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:43.903347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:43.903347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:43.903347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:43.903347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:18:43.903347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:18:43.903347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:18:43.904378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:18:43.904378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:43.904378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:43.904378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:43.904378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:43.904378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:43.904378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:43.904378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:43.904378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:18:43.904378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:43.904378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:43.904378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:43.904378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:43.904378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:43.904378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:43.904378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:43.904378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:43.905837 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:18:43.905837 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:18:43.905837 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:18:43.905837 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:18:43.905837 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 619/1237] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:18:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:43.910671 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c74c2f-8b7e-4d88-a3aa-e27e09b206fb None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:44.008466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c74c2f-8b7e-4d88-a3aa-e27e09b206fb None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:44.017425 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:44.041272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82be085-262b-4557-9bc1-5ae52413269e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:44.041913 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82be085-262b-4557-9bc1-5ae52413269e None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:44.042190 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82be085-262b-4557-9bc1-5ae52413269e None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:44.042460 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82be085-262b-4557-9bc1-5ae52413269e None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:44.043197 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82be085-262b-4557-9bc1-5ae52413269e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:44.043498 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82be085-262b-4557-9bc1-5ae52413269e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:44.044849 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82be085-262b-4557-9bc1-5ae52413269e None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:44.055882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 619/1238] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:44.061701 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d499b0b-84ae-450d-b449-fc11a613d01b None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:44.161226 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d499b0b-84ae-450d-b449-fc11a613d01b None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:44.194013 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b563d-6542-45b3-ba65-6e81ebf862fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:44.194861 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b563d-6542-45b3-ba65-6e81ebf862fc None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:44.195146 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b563d-6542-45b3-ba65-6e81ebf862fc None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:44.195421 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b563d-6542-45b3-ba65-6e81ebf862fc None admin] PATH_INFO: `/v3/users/placement` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:44.198067 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:44.203464 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b563d-6542-45b3-ba65-6e81ebf862fc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:44.204888 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b563d-6542-45b3-ba65-6e81ebf862fc None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:44.221946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-219b563d-6542-45b3-ba65-6e81ebf862fc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 03:18:44.221946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:44.221946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:44.221946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:44.221946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:44.221946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:44.221946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:44.221946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:44.221946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:44.221946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:44.221946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:44.221946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:44.221946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:18:44.221946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:18:44.221946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:18:44.221946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:18:44.223108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:44.223108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:44.223108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:18:44.223108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:18:44.223108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:18:44.223108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:18:44.223108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:44.223108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:44.223108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:44.223108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:44.223108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:44.223108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:18:44.223108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:44.223108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:44.223108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:44.223108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:44.223108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:44.223108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:44.224308 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:44.224308 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:44.224308 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:18:44.224308 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:18:44.224308 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:18:44.224308 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:18:44.224308 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:18:44.224308 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:18:44.224308 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 03:18:44.224308 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:18:44.224308 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 620/1239] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:18:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:44.228971 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82be085-262b-4557-9bc1-5ae52413269e None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:44.333003 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82be085-262b-4557-9bc1-5ae52413269e None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:44.365388 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793d858-e463-4ef2-bab0-57f639ff3437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:44.366048 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793d858-e463-4ef2-bab0-57f639ff3437 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:44.366326 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793d858-e463-4ef2-bab0-57f639ff3437 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:44.366597 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793d858-e463-4ef2-bab0-57f639ff3437 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:44.367441 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793d858-e463-4ef2-bab0-57f639ff3437 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:44.367745 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793d858-e463-4ef2-bab0-57f639ff3437 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:44.369151 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793d858-e463-4ef2-bab0-57f639ff3437 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:44.386107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 620/1240] 10.222.0.24 () {56 vars in 1143 bytes} [Fri May 27 03:18:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:44.392506 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b563d-6542-45b3-ba65-6e81ebf862fc None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:44.493180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b563d-6542-45b3-ba65-6e81ebf862fc None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:44.526845 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8a0e4-93a3-437e-aec4-4eca83043dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:44.527499 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8a0e4-93a3-437e-aec4-4eca83043dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:44.527782 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8a0e4-93a3-437e-aec4-4eca83043dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:44.528058 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8a0e4-93a3-437e-aec4-4eca83043dc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:44.591581 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:44.598543 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c8a0e4-93a3-437e-aec4-4eca83043dc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:44.600032 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c8a0e4-93a3-437e-aec4-4eca83043dc5 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:44.697213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 621/1241] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:18:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:18:44.703910 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793d858-e463-4ef2-bab0-57f639ff3437 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:44.730561 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:44.799436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793d858-e463-4ef2-bab0-57f639ff3437 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:44.832074 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b06e95-13f9-48ad-a629-546473cb3d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:44.832914 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b06e95-13f9-48ad-a629-546473cb3d5c None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:44.833194 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b06e95-13f9-48ad-a629-546473cb3d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:44.833468 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b06e95-13f9-48ad-a629-546473cb3d5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:44.843212 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b06e95-13f9-48ad-a629-546473cb3d5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:44.844656 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b06e95-13f9-48ad-a629-546473cb3d5c None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:44.862045 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-b4b06e95-13f9-48ad-a629-546473cb3d5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:18:44.862045 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:44.862045 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:44.862045 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:44.862045 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:44.862045 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:44.862045 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:44.862045 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:44.862045 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:44.862045 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:44.862045 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:44.862045 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:44.862045 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:18:44.862045 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:18:44.862045 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:18:44.863106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:18:44.863106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:44.863106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:44.863106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:44.863106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:44.863106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:44.863106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:44.863106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:44.863106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:18:44.863106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:44.863106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:44.863106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:44.863106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:44.863106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:44.863106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:44.863106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:44.863106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:44.864068 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:18:44.864068 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:18:44.864068 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:18:44.864068 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:18:44.864068 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 621/1242] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:18:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:44.868935 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8a0e4-93a3-437e-aec4-4eca83043dc5 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:44.965838 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:44.971645 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8a0e4-93a3-437e-aec4-4eca83043dc5 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:45.005565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8f9bf-4683-4161-a799-d4c93d2a292c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:45.006221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8f9bf-4683-4161-a799-d4c93d2a292c None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:45.006549 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8f9bf-4683-4161-a799-d4c93d2a292c None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:45.007062 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8f9bf-4683-4161-a799-d4c93d2a292c None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:45.007962 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8f9bf-4683-4161-a799-d4c93d2a292c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:45.009185 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8f9bf-4683-4161-a799-d4c93d2a292c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:45.010555 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8f9bf-4683-4161-a799-d4c93d2a292c None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:45.021905 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 622/1243] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:45.027542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b06e95-13f9-48ad-a629-546473cb3d5c None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:45.104956 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:45.125516 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b06e95-13f9-48ad-a629-546473cb3d5c None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:45.157545 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dca67e-9991-42fe-b890-e10308f3a8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:45.158326 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dca67e-9991-42fe-b890-e10308f3a8fe None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:45.158611 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dca67e-9991-42fe-b890-e10308f3a8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:45.158908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dca67e-9991-42fe-b890-e10308f3a8fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:45.165977 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dca67e-9991-42fe-b890-e10308f3a8fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:45.167471 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dca67e-9991-42fe-b890-e10308f3a8fe None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:45.181497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-e3dca67e-9991-42fe-b890-e10308f3a8fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:18:45.181497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:45.181497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:45.181497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:45.181497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:45.181497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:45.181497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:45.181497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:45.181497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:45.181497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:45.181497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:45.181497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:45.181497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:18:45.181497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:18:45.181497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:18:45.182551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:18:45.182551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:45.182551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:45.182551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:45.182551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:45.182551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:45.182551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:45.182551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:45.182551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:18:45.182551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:45.182551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:45.182551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:45.182551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:45.182551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:45.182551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:45.182551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:45.182551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:45.183692 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:18:45.183692 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:18:45.183692 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:18:45.183692 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:18:45.183692 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:18:45.183692 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:18:45.183692 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:18:45.183692 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:18:45.183692 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 622/1244] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:18:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:45.189049 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8f9bf-4683-4161-a799-d4c93d2a292c None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:45.292074 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8f9bf-4683-4161-a799-d4c93d2a292c None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:45.325064 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356a19fe-08fa-48d6-914a-abeb7e7df8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:45.325707 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a19fe-08fa-48d6-914a-abeb7e7df8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:45.325984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a19fe-08fa-48d6-914a-abeb7e7df8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:45.326253 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356a19fe-08fa-48d6-914a-abeb7e7df8e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:45.327648 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356a19fe-08fa-48d6-914a-abeb7e7df8e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:45.327902 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356a19fe-08fa-48d6-914a-abeb7e7df8e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:45.329316 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356a19fe-08fa-48d6-914a-abeb7e7df8e3 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:45.340904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 623/1245] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:18:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:18:45.347094 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dca67e-9991-42fe-b890-e10308f3a8fe None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:45.444891 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dca67e-9991-42fe-b890-e10308f3a8fe None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:45.478754 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25511635-02ca-498f-b507-0abaf5753fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:45.479418 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25511635-02ca-498f-b507-0abaf5753fcb None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:45.479739 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25511635-02ca-498f-b507-0abaf5753fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:45.480017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25511635-02ca-498f-b507-0abaf5753fcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:45.480913 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25511635-02ca-498f-b507-0abaf5753fcb None admin] RBAC: Adding query filter params (role.id=8e315ffd561a48f0891ef53b51d548d8, scope.project.id=6ceffdd955324aec84ba048d2137ae00, user.id=95f2d0aceef4451a83ee89fac5130a0c) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:45.481221 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25511635-02ca-498f-b507-0abaf5753fcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:45.482644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25511635-02ca-498f-b507-0abaf5753fcb None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:45.488781 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 623/1246] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:18:45 2022] GET /identity/v3/role_assignments?user.id=95f2d0aceef4451a83ee89fac5130a0c&scope.project.id=6ceffdd955324aec84ba048d2137ae00&role.id=8e315ffd561a48f0891ef53b51d548d8 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:47.714075 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba27d3-652a-4ca9-b92f-3f609160b5fe None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:47.714625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba27d3-652a-4ca9-b92f-3f609160b5fe None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:47.714730 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba27d3-652a-4ca9-b92f-3f609160b5fe None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:47.716137 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 624/1247] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:18:47.721306 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05fa59-32b2-4611-8f74-ac9ea55606e2 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:47.721721 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05fa59-32b2-4611-8f74-ac9ea55606e2 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:47.721856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05fa59-32b2-4611-8f74-ac9ea55606e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:47.726865 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:47.739137 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-4c05fa59-32b2-4611-8f74-ac9ea55606e2 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:18:47.740304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-4c05fa59-32b2-4611-8f74-ac9ea55606e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:18:47.803098 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 624/1248] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:18:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:18:47.826574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba27d3-652a-4ca9-b92f-3f609160b5fe None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:47.844308 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:47.931945 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba27d3-652a-4ca9-b92f-3f609160b5fe None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:47.965057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da7abf0-0ba6-4ee0-aa69-7ea2afab7474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:47.965766 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7abf0-0ba6-4ee0-aa69-7ea2afab7474 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:47.966046 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7abf0-0ba6-4ee0-aa69-7ea2afab7474 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:47.966350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7abf0-0ba6-4ee0-aa69-7ea2afab7474 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:47.976105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da7abf0-0ba6-4ee0-aa69-7ea2afab7474 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:18:47.984249 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-1da7abf0-0ba6-4ee0-aa69-7ea2afab7474 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:18:47.984249 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:47.984249 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:47.984249 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:47.984249 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:47.984249 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:47.984249 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:47.984249 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:47.984249 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:47.984249 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:47.984249 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:47.984249 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:47.984249 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:18:47.984249 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:18:47.984249 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:18:47.984249 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:18:47.991621 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:18:47.991621 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:18:47.991621 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:18:47.991621 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:18:47.991621 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:47.991621 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:47.991621 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:47.991621 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:47.991621 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:47.991621 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:47.991621 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:47.991621 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:18:47.991621 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:47.991621 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:47.991621 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:47.991621 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:47.992620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:47.992620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:47.992620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:47.992620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:47.992620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:18:47.992620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:18:47.992620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:18:47.992620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:18:47.992620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:18:47.992620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:18:47.992620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:18:47.992620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:18:47.992620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 625/1249] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:18:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:47.993319 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05fa59-32b2-4611-8f74-ac9ea55606e2 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:48.088187 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05fa59-32b2-4611-8f74-ac9ea55606e2 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:48.117619 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:48.122518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc844aca-c0ed-4c8c-9198-26f576fed486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:48.123223 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc844aca-c0ed-4c8c-9198-26f576fed486 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:48.124078 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc844aca-c0ed-4c8c-9198-26f576fed486 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:48.124362 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc844aca-c0ed-4c8c-9198-26f576fed486 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:48.125113 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc844aca-c0ed-4c8c-9198-26f576fed486 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:48.125424 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc844aca-c0ed-4c8c-9198-26f576fed486 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:48.126918 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc844aca-c0ed-4c8c-9198-26f576fed486 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:48.134820 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 625/1250] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:18:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:48.141017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da7abf0-0ba6-4ee0-aa69-7ea2afab7474 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:48.225654 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:48.247577 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da7abf0-0ba6-4ee0-aa69-7ea2afab7474 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:48.280228 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b787f-5905-4fee-a068-16c3a305a67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:48.281023 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b787f-5905-4fee-a068-16c3a305a67e None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:48.281305 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b787f-5905-4fee-a068-16c3a305a67e None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:48.281578 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b787f-5905-4fee-a068-16c3a305a67e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:48.291808 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0b787f-5905-4fee-a068-16c3a305a67e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:48.293252 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0b787f-5905-4fee-a068-16c3a305a67e None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:48.310354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-5c0b787f-5905-4fee-a068-16c3a305a67e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:18:48.310354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:48.310354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:48.310354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:48.310354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:48.310354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:48.310354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:48.310354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:48.310354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:48.310354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:48.310354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:48.310354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:48.310354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:18:48.310354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:18:48.310354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:18:48.311477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:18:48.311477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:48.311477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:48.311477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:48.311477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:48.311477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:48.311477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:48.311477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:48.311477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:18:48.311477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:48.311477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:48.311477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:48.311477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:48.311477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:48.311477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:48.311477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:48.311477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:48.312423 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:18:48.312423 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:18:48.312423 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:18:48.312423 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:18:48.312423 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 626/1251] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:18:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:48.317427 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc844aca-c0ed-4c8c-9198-26f576fed486 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:48.414671 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc844aca-c0ed-4c8c-9198-26f576fed486 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:48.447369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd064e6-da2e-409b-8269-a21bca9bfba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:48.448025 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd064e6-da2e-409b-8269-a21bca9bfba9 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:48.448306 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd064e6-da2e-409b-8269-a21bca9bfba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:48.448580 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd064e6-da2e-409b-8269-a21bca9bfba9 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:48.449553 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd064e6-da2e-409b-8269-a21bca9bfba9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:48.449859 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd064e6-da2e-409b-8269-a21bca9bfba9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:48.451255 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd064e6-da2e-409b-8269-a21bca9bfba9 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:48.469552 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 626/1252] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:48.479372 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b787f-5905-4fee-a068-16c3a305a67e None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:48.580539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b787f-5905-4fee-a068-16c3a305a67e None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:48.605424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:48.613435 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac24c92-325e-460a-863d-669ad10c7aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:48.614234 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac24c92-325e-460a-863d-669ad10c7aaa None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:48.614564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac24c92-325e-460a-863d-669ad10c7aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:48.614782 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac24c92-325e-460a-863d-669ad10c7aaa None admin] PATH_INFO: `/v3/users/placement` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:48.622010 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac24c92-325e-460a-863d-669ad10c7aaa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:48.623522 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac24c92-325e-460a-863d-669ad10c7aaa None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:48.637552 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-4ac24c92-325e-460a-863d-669ad10c7aaa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 03:18:48.637552 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:48.637552 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:48.637552 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:48.637552 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:48.637552 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:48.637552 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:48.637552 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:48.637552 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:48.637552 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:48.637552 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:48.637552 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:48.637552 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:18:48.637552 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:18:48.637552 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:18:48.637552 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:18:48.638675 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:48.638675 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:48.638675 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:18:48.638675 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:18:48.638675 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:18:48.638675 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:18:48.638675 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:48.638675 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:48.638675 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:48.638675 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:48.638675 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:48.638675 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:18:48.638675 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:48.638675 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:48.638675 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:48.638675 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:48.638675 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:48.638675 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:48.640625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:48.640625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:48.640625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:18:48.640625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:18:48.640625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:18:48.640625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:18:48.640625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:18:48.640625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:18:48.640625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 03:18:48.640625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:18:48.640625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 627/1253] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:18:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:48.645011 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd064e6-da2e-409b-8269-a21bca9bfba9 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:48.698950 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:48.741855 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd064e6-da2e-409b-8269-a21bca9bfba9 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:48.774159 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75da71e-76bd-4f31-9218-49d6365b0905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:48.774810 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75da71e-76bd-4f31-9218-49d6365b0905 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:48.775124 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75da71e-76bd-4f31-9218-49d6365b0905 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:48.775399 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75da71e-76bd-4f31-9218-49d6365b0905 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:48.776224 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75da71e-76bd-4f31-9218-49d6365b0905 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:48.776537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75da71e-76bd-4f31-9218-49d6365b0905 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:48.777974 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75da71e-76bd-4f31-9218-49d6365b0905 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:48.794397 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 627/1254] 10.222.0.24 () {56 vars in 1143 bytes} [Fri May 27 03:18:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:48.800587 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac24c92-325e-460a-863d-669ad10c7aaa None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:48.899525 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac24c92-325e-460a-863d-669ad10c7aaa None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:48.931637 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1f1d7-e0bb-47c0-b429-ad13b0961cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:48.932264 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1f1d7-e0bb-47c0-b429-ad13b0961cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:48.932543 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1f1d7-e0bb-47c0-b429-ad13b0961cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:48.932815 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1f1d7-e0bb-47c0-b429-ad13b0961cb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:48.986187 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:48.997885 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd1f1d7-e0bb-47c0-b429-ad13b0961cb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:48.999804 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd1f1d7-e0bb-47c0-b429-ad13b0961cb2 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:49.096396 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 628/1255] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:18:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:18:49.102384 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75da71e-76bd-4f31-9218-49d6365b0905 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:49.201441 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75da71e-76bd-4f31-9218-49d6365b0905 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:49.215490 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:49.236814 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e97ff-320b-45ad-b4e9-cd5cc07729e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:49.237602 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e97ff-320b-45ad-b4e9-cd5cc07729e0 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:49.237883 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e97ff-320b-45ad-b4e9-cd5cc07729e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:49.238158 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e97ff-320b-45ad-b4e9-cd5cc07729e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:49.247994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e97ff-320b-45ad-b4e9-cd5cc07729e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:49.249944 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e97ff-320b-45ad-b4e9-cd5cc07729e0 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:49.266749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-dd2e97ff-320b-45ad-b4e9-cd5cc07729e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:18:49.266749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:49.266749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:49.266749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:49.266749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:49.266749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:49.266749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:49.266749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:49.266749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:49.266749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:49.266749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:49.266749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:49.266749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:18:49.266749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:18:49.266749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:18:49.267813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:18:49.267813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:49.267813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:49.267813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:49.267813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:49.267813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:49.267813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:49.267813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:49.267813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:18:49.267813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:49.267813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:49.267813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:49.267813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:49.267813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:49.267813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:49.267813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:49.267813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:49.268834 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:18:49.268834 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:18:49.268834 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:18:49.268834 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:18:49.268834 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 628/1256] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:18:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:49.273952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1f1d7-e0bb-47c0-b429-ad13b0961cb2 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:49.359330 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:49.370742 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1f1d7-e0bb-47c0-b429-ad13b0961cb2 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:49.404589 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa0013-7f6c-45a9-8202-05196fa6ae48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:49.405789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa0013-7f6c-45a9-8202-05196fa6ae48 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:49.406073 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa0013-7f6c-45a9-8202-05196fa6ae48 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:49.406348 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa0013-7f6c-45a9-8202-05196fa6ae48 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:49.407095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfa0013-7f6c-45a9-8202-05196fa6ae48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:49.407404 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfa0013-7f6c-45a9-8202-05196fa6ae48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:49.408764 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfa0013-7f6c-45a9-8202-05196fa6ae48 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:49.419684 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 629/1257] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:18:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:49.425387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e97ff-320b-45ad-b4e9-cd5cc07729e0 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:49.522477 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e97ff-320b-45ad-b4e9-cd5cc07729e0 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:49.555468 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6fdc9-edf6-4900-856f-ae02eadccdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:49.556244 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6fdc9-edf6-4900-856f-ae02eadccdf4 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:49.556520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6fdc9-edf6-4900-856f-ae02eadccdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:49.556793 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6fdc9-edf6-4900-856f-ae02eadccdf4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:49.563755 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac6fdc9-edf6-4900-856f-ae02eadccdf4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:49.565244 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac6fdc9-edf6-4900-856f-ae02eadccdf4 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:49.578872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-3ac6fdc9-edf6-4900-856f-ae02eadccdf4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:18:49.578872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:49.578872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:49.578872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:49.578872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:49.578872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:49.578872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:49.578872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:49.578872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:49.578872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:49.578872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:49.578872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:49.578872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:18:49.578872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:18:49.578872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:18:49.582049 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:18:49.582049 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:49.582049 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:49.582049 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:49.582049 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:49.582049 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:49.582049 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:49.582049 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:49.582049 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:18:49.582049 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:49.582049 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:49.582049 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:49.582049 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:49.582049 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:49.582049 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:49.582049 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:49.582049 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:49.583272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:18:49.583272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:18:49.583272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:18:49.583272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:18:49.583272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:18:49.583272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:18:49.583272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:18:49.583272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:18:49.583272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 629/1258] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:18:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:49.585857 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa0013-7f6c-45a9-8202-05196fa6ae48 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:49.684890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa0013-7f6c-45a9-8202-05196fa6ae48 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:49.718441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac497e9-6fe9-4c0a-9714-cea3bd3d3447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:49.719117 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac497e9-6fe9-4c0a-9714-cea3bd3d3447 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:49.719398 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac497e9-6fe9-4c0a-9714-cea3bd3d3447 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:49.719669 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac497e9-6fe9-4c0a-9714-cea3bd3d3447 None admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:49.720443 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac497e9-6fe9-4c0a-9714-cea3bd3d3447 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:49.720747 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac497e9-6fe9-4c0a-9714-cea3bd3d3447 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:49.722156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac497e9-6fe9-4c0a-9714-cea3bd3d3447 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:49.735312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 630/1259] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:18:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:49.741676 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6fdc9-edf6-4900-856f-ae02eadccdf4 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:49.767278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:49.838914 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6fdc9-edf6-4900-856f-ae02eadccdf4 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:49.871604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124f188-2c05-4361-b1fb-e7f63f859c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:49.872266 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124f188-2c05-4361-b1fb-e7f63f859c62 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:49.872546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124f188-2c05-4361-b1fb-e7f63f859c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:49.872818 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124f188-2c05-4361-b1fb-e7f63f859c62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:49.874166 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8124f188-2c05-4361-b1fb-e7f63f859c62 None admin] RBAC: Adding query filter params (role.id=e5a908bd3ed5445b82a6053246533b88, scope.project.id=6ceffdd955324aec84ba048d2137ae00, user.id=95f2d0aceef4451a83ee89fac5130a0c) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:49.874463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8124f188-2c05-4361-b1fb-e7f63f859c62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:49.875936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8124f188-2c05-4361-b1fb-e7f63f859c62 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:49.881674 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 630/1260] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:18:49 2022] GET /identity/v3/role_assignments?user.id=95f2d0aceef4451a83ee89fac5130a0c&scope.project.id=6ceffdd955324aec84ba048d2137ae00&role.id=e5a908bd3ed5445b82a6053246533b88 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:52.056708 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9affd-ffb4-4f15-96c2-016e891c67c7 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:52.057261 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9affd-ffb4-4f15-96c2-016e891c67c7 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:52.057261 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9affd-ffb4-4f15-96c2-016e891c67c7 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:52.058610 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 631/1261] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:18:52.063822 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b184a2-79eb-4055-9900-098b9d8cee89 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:52.064248 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b184a2-79eb-4055-9900-098b9d8cee89 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:52.064397 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b184a2-79eb-4055-9900-098b9d8cee89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:52.081496 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-75b184a2-79eb-4055-9900-098b9d8cee89 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:18:52.082674 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-75b184a2-79eb-4055-9900-098b9d8cee89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:18:52.146508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 631/1262] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:18:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:18:52.168731 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9affd-ffb4-4f15-96c2-016e891c67c7 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:52.180380 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:52.270431 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9affd-ffb4-4f15-96c2-016e891c67c7 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:52.302610 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7738b04-3cd8-4350-bc4c-7171cb7f445a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:52.303407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7738b04-3cd8-4350-bc4c-7171cb7f445a None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:52.303685 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7738b04-3cd8-4350-bc4c-7171cb7f445a None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:52.303957 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7738b04-3cd8-4350-bc4c-7171cb7f445a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:52.313647 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7738b04-3cd8-4350-bc4c-7171cb7f445a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:18:52.321108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-b7738b04-3cd8-4350-bc4c-7171cb7f445a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:18:52.321108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:52.321108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:52.321108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:52.321108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:52.321108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:52.321108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:52.321108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:52.321108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:52.321108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:52.321108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:52.321108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:52.321108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:18:52.321108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:18:52.321108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:18:52.321108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:18:52.322281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:18:52.322281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:18:52.322281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:18:52.322281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:18:52.322281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:52.322281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:52.322281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:52.322281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:52.322281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:52.322281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:52.322281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:52.322281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:18:52.322281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:52.322281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:52.322281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:52.322281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:52.323277 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:52.323277 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:52.323277 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:52.323277 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:52.323277 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:18:52.323277 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:18:52.323277 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:18:52.323277 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:18:52.323277 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:18:52.323277 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:18:52.323277 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:18:52.323277 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:18:52.323277 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 632/1263] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:18:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:52.329035 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b184a2-79eb-4055-9900-098b9d8cee89 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:52.347211 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:52.422730 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b184a2-79eb-4055-9900-098b9d8cee89 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:52.455948 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceadd98-ef29-4db7-ba02-f2262da3b345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:52.456603 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceadd98-ef29-4db7-ba02-f2262da3b345 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:52.456885 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceadd98-ef29-4db7-ba02-f2262da3b345 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:52.457156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceadd98-ef29-4db7-ba02-f2262da3b345 None admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:52.457895 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceadd98-ef29-4db7-ba02-f2262da3b345 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:52.458199 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceadd98-ef29-4db7-ba02-f2262da3b345 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:52.459583 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceadd98-ef29-4db7-ba02-f2262da3b345 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:52.467909 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 632/1264] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:18:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:52.473591 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7738b04-3cd8-4350-bc4c-7171cb7f445a None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:52.545919 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:52.579506 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7738b04-3cd8-4350-bc4c-7171cb7f445a None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:52.612917 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a6d7b-b035-4de2-9d8b-3e8000cd1188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:52.613700 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a6d7b-b035-4de2-9d8b-3e8000cd1188 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:52.613989 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a6d7b-b035-4de2-9d8b-3e8000cd1188 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:52.614261 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6a6d7b-b035-4de2-9d8b-3e8000cd1188 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:52.624544 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a6d7b-b035-4de2-9d8b-3e8000cd1188 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:52.625992 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6a6d7b-b035-4de2-9d8b-3e8000cd1188 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:52.642566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-7e6a6d7b-b035-4de2-9d8b-3e8000cd1188 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:18:52.642566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:52.642566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:52.642566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:52.642566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:52.642566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:52.642566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:52.642566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:52.642566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:52.642566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:52.642566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:52.642566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:52.642566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:18:52.642566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:18:52.642566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:18:52.643651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:18:52.643651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:52.643651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:52.643651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:52.643651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:52.643651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:52.643651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:52.643651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:52.643651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:18:52.643651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:52.643651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:52.643651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:52.643651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:52.643651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:52.643651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:52.643651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:52.643651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:52.644637 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:18:52.644637 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:18:52.644637 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:18:52.644637 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:18:52.644637 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 633/1265] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:18:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:52.649550 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceadd98-ef29-4db7-ba02-f2262da3b345 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:52.729826 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:52.747741 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceadd98-ef29-4db7-ba02-f2262da3b345 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:52.780500 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0fc30-341f-422c-b34c-ac6fcdaea30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:52.781154 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0fc30-341f-422c-b34c-ac6fcdaea30d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:52.781435 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0fc30-341f-422c-b34c-ac6fcdaea30d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:52.781710 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0fc30-341f-422c-b34c-ac6fcdaea30d None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:52.782423 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc0fc30-341f-422c-b34c-ac6fcdaea30d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:52.782729 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc0fc30-341f-422c-b34c-ac6fcdaea30d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:52.784170 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc0fc30-341f-422c-b34c-ac6fcdaea30d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:52.794822 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 633/1266] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:18:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:52.804641 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a6d7b-b035-4de2-9d8b-3e8000cd1188 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:52.903656 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6a6d7b-b035-4de2-9d8b-3e8000cd1188 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:52.922368 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:52.936421 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a4c943-2cd5-478c-8c20-48f632f31720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:52.937227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4c943-2cd5-478c-8c20-48f632f31720 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:52.937509 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4c943-2cd5-478c-8c20-48f632f31720 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:52.937783 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4c943-2cd5-478c-8c20-48f632f31720 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:52.945061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a4c943-2cd5-478c-8c20-48f632f31720 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:52.946492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a4c943-2cd5-478c-8c20-48f632f31720 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:52.961464 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-38a4c943-2cd5-478c-8c20-48f632f31720 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 03:18:52.961464 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:52.961464 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:52.961464 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:52.961464 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:52.961464 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:52.961464 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:52.961464 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:52.961464 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:52.961464 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:52.961464 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:52.961464 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:52.961464 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:18:52.961464 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:18:52.961464 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:18:52.961464 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:18:52.962569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:52.962569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:52.962569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:18:52.962569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:18:52.962569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:18:52.962569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:18:52.962569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:52.962569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:52.962569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:52.962569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:52.962569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:52.962569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:18:52.962569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:52.962569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:52.962569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:52.962569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:52.962569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:52.962569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:52.968409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:52.968409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:52.968409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:18:52.968409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:18:52.968409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:18:52.968409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:18:52.968409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:18:52.968409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:18:52.968409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 03:18:52.968409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:18:52.968409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 634/1267] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:18:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:52.969143 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0fc30-341f-422c-b34c-ac6fcdaea30d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:53.067531 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0fc30-341f-422c-b34c-ac6fcdaea30d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:53.099547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de913e5-fadc-4071-8b41-ade776e03b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:53.100212 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de913e5-fadc-4071-8b41-ade776e03b76 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:53.100490 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de913e5-fadc-4071-8b41-ade776e03b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:53.100762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de913e5-fadc-4071-8b41-ade776e03b76 None admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:53.101579 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de913e5-fadc-4071-8b41-ade776e03b76 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:53.101889 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de913e5-fadc-4071-8b41-ade776e03b76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:53.103341 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de913e5-fadc-4071-8b41-ade776e03b76 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:53.112503 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:53.120532 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 634/1268] 10.222.0.24 () {56 vars in 1143 bytes} [Fri May 27 03:18:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:18:53.126798 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a4c943-2cd5-478c-8c20-48f632f31720 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:53.222509 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a4c943-2cd5-478c-8c20-48f632f31720 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:53.254892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30cb51-b250-4db3-b5a9-b321a3fcf76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:53.262196 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30cb51-b250-4db3-b5a9-b321a3fcf76b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:53.262196 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30cb51-b250-4db3-b5a9-b321a3fcf76b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:53.262196 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30cb51-b250-4db3-b5a9-b321a3fcf76b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:53.266296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30cb51-b250-4db3-b5a9-b321a3fcf76b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:53.267823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30cb51-b250-4db3-b5a9-b321a3fcf76b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:53.284407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-fb30cb51-b250-4db3-b5a9-b321a3fcf76b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:18:53.284407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:53.284407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:53.284407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:53.284407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:53.284407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:53.284407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:53.284407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:53.284407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:53.284407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:53.284407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:53.284407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:53.284407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:18:53.284407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:18:53.284407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:18:53.285274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:18:53.285274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:53.285274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:53.285274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:53.285274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:53.285274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:53.285274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:53.285274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:53.285274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:18:53.285274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:53.285274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:53.285274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:53.285274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:53.285274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:53.285274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:53.285274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:53.285274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:53.286177 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:18:53.286177 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:18:53.286177 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:18:53.286177 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:18:53.286177 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 635/1269] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:18:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:53.291391 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de913e5-fadc-4071-8b41-ade776e03b76 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:53.390573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de913e5-fadc-4071-8b41-ade776e03b76 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:53.424112 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1467784-4115-4da1-9d75-83f48102efc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:53.424763 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1467784-4115-4da1-9d75-83f48102efc2 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:53.425044 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1467784-4115-4da1-9d75-83f48102efc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:53.425318 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1467784-4115-4da1-9d75-83f48102efc2 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:53.426035 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1467784-4115-4da1-9d75-83f48102efc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:53.426339 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1467784-4115-4da1-9d75-83f48102efc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:53.427792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1467784-4115-4da1-9d75-83f48102efc2 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:53.438461 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 635/1270] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:53.444099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30cb51-b250-4db3-b5a9-b321a3fcf76b None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:53.476451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:53.544765 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30cb51-b250-4db3-b5a9-b321a3fcf76b None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:53.578327 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9afacd-0ecb-473f-969e-f155af935890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:53.579141 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9afacd-0ecb-473f-969e-f155af935890 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:53.579424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9afacd-0ecb-473f-969e-f155af935890 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:53.579696 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9afacd-0ecb-473f-969e-f155af935890 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:53.587522 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9afacd-0ecb-473f-969e-f155af935890 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:53.589109 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9afacd-0ecb-473f-969e-f155af935890 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:53.610668 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-9f9afacd-0ecb-473f-969e-f155af935890 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:18:53.610668 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:53.610668 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:53.610668 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:53.610668 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:53.610668 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:53.610668 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:53.610668 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:53.610668 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:53.610668 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:53.610668 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:53.610668 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:53.610668 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:18:53.610668 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:18:53.610668 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:18:53.611930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:18:53.611930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:53.611930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:53.611930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:53.611930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:53.611930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:53.611930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:53.611930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:53.611930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:18:53.611930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:53.611930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:53.611930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:53.611930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:53.611930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:53.611930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:53.611930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:53.611930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:53.613179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:18:53.613179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:18:53.613179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:18:53.613179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:18:53.613179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:18:53.613179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:18:53.613179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:18:53.613179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:18:53.613179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 636/1271] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:18:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:53.617949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1467784-4115-4da1-9d75-83f48102efc2 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:53.675901 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:53.716276 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1467784-4115-4da1-9d75-83f48102efc2 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:53.748576 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba6a91-0a81-4b2a-9932-ad7c1f5acc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:53.749169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba6a91-0a81-4b2a-9932-ad7c1f5acc2d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:53.749535 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba6a91-0a81-4b2a-9932-ad7c1f5acc2d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:53.749807 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba6a91-0a81-4b2a-9932-ad7c1f5acc2d None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:53.750588 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ba6a91-0a81-4b2a-9932-ad7c1f5acc2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:53.750935 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ba6a91-0a81-4b2a-9932-ad7c1f5acc2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:53.752363 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ba6a91-0a81-4b2a-9932-ad7c1f5acc2d None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:53.764613 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 636/1272] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:18:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:53.770297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9afacd-0ecb-473f-969e-f155af935890 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:53.856722 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:53.866567 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9afacd-0ecb-473f-969e-f155af935890 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:53.900023 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88d6fc-bf3c-41f7-b19d-a881cbdce182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:53.900997 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88d6fc-bf3c-41f7-b19d-a881cbdce182 None admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:53.901241 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88d6fc-bf3c-41f7-b19d-a881cbdce182 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:53.901510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88d6fc-bf3c-41f7-b19d-a881cbdce182 None admin] PATH_INFO: `/v3/projects/6ceffdd955324aec84ba048d2137ae00/users/95f2d0aceef4451a83ee89fac5130a0c/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:53.903448 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c88d6fc-bf3c-41f7-b19d-a881cbdce182 None admin] RBAC: Authorizing `identity:create_grant(project_id=6ceffdd955324aec84ba048d2137ae00, user_id=95f2d0aceef4451a83ee89fac5130a0c, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:53.904880 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c88d6fc-bf3c-41f7-b19d-a881cbdce182 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:54.111811 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 637/1273] 10.222.0.24 () {58 vars in 1439 bytes} [Fri May 27 03:18:53 2022] PUT /identity/v3/projects/6ceffdd955324aec84ba048d2137ae00/users/95f2d0aceef4451a83ee89fac5130a0c/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:18:56.292759 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25232718-383f-40ba-a890-e2d0fd47e95c None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:56.293331 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25232718-383f-40ba-a890-e2d0fd47e95c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:56.293331 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25232718-383f-40ba-a890-e2d0fd47e95c None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:56.294664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 637/1274] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:18:56.300356 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a7163-a206-41d3-92b3-aaad8d6848e1 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:56.300767 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a7163-a206-41d3-92b3-aaad8d6848e1 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:56.300908 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a7163-a206-41d3-92b3-aaad8d6848e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:56.318122 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-569a7163-a206-41d3-92b3-aaad8d6848e1 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:18:56.319341 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-569a7163-a206-41d3-92b3-aaad8d6848e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:18:56.384761 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 638/1275] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:18:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:18:56.407968 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25232718-383f-40ba-a890-e2d0fd47e95c None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:56.505812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25232718-383f-40ba-a890-e2d0fd47e95c None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:56.522572 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:56.539821 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a072a-53ec-4527-b1cd-5210108a735d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:56.540603 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a072a-53ec-4527-b1cd-5210108a735d None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:56.540882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a072a-53ec-4527-b1cd-5210108a735d None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:56.541150 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28a072a-53ec-4527-b1cd-5210108a735d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:56.551567 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28a072a-53ec-4527-b1cd-5210108a735d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:18:56.559107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-d28a072a-53ec-4527-b1cd-5210108a735d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:18:56.559107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:56.559107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:56.559107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:56.559107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:56.559107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:56.559107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:56.559107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:56.559107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:56.559107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:56.559107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:56.559107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:56.559107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:18:56.559107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:18:56.559107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:18:56.559107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:18:56.560307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:18:56.560307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:18:56.560307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:18:56.560307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:18:56.560307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:56.560307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:56.560307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:56.560307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:56.560307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:56.560307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:56.560307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:56.560307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:18:56.560307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:56.560307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:56.560307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:56.560307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:56.561434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:56.561434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:56.561434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:56.561434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:56.561434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:18:56.561434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:18:56.561434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:18:56.561434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:18:56.561434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:18:56.561434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:18:56.561434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:18:56.561434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:18:56.561434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 638/1276] 10.222.0.24 () {56 vars in 1096 bytes} [Fri May 27 03:18:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:56.566407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a7163-a206-41d3-92b3-aaad8d6848e1 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:56.658790 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:56.676653 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569a7163-a206-41d3-92b3-aaad8d6848e1 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:56.710376 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b5588-d605-423c-8736-c5717b0954a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:56.711080 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b5588-d605-423c-8736-c5717b0954a4 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:56.711366 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b5588-d605-423c-8736-c5717b0954a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:56.711641 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b5588-d605-423c-8736-c5717b0954a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:56.712418 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8b5588-d605-423c-8736-c5717b0954a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:56.712728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8b5588-d605-423c-8736-c5717b0954a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:56.714227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8b5588-d605-423c-8736-c5717b0954a4 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:56.723124 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 639/1277] 10.222.0.24 () {56 vars in 1099 bytes} [Fri May 27 03:18:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:56.728876 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a072a-53ec-4527-b1cd-5210108a735d None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:56.828013 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28a072a-53ec-4527-b1cd-5210108a735d None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:56.863181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278e47d-ef3d-484b-9812-ae01c01072a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:56.863990 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278e47d-ef3d-484b-9812-ae01c01072a1 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:56.864269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278e47d-ef3d-484b-9812-ae01c01072a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:56.864568 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278e47d-ef3d-484b-9812-ae01c01072a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:56.875644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5278e47d-ef3d-484b-9812-ae01c01072a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:56.877082 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5278e47d-ef3d-484b-9812-ae01c01072a1 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:56.893624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-5278e47d-ef3d-484b-9812-ae01c01072a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:18:56.893624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:56.893624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:56.893624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:56.893624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:56.893624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:56.893624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:56.893624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:56.893624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:56.893624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:56.893624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:56.893624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:56.893624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:18:56.893624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:18:56.893624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:18:56.894542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:18:56.894542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:56.894542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:56.894542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:56.894542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:56.894542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:56.894542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:56.894542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:56.894542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:18:56.894542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:56.894542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:56.894542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:56.894542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:56.894542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:56.894542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:56.894542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:56.894542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:56.895598 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:18:56.895598 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:18:56.895598 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:18:56.895598 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:18:56.895598 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 639/1278] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:18:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:56.900655 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b5588-d605-423c-8736-c5717b0954a4 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:57.009096 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b5588-d605-423c-8736-c5717b0954a4 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:57.047384 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82590d98-8d75-4aa0-a752-3c5080b49ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:57.048042 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82590d98-8d75-4aa0-a752-3c5080b49ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:57.048323 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82590d98-8d75-4aa0-a752-3c5080b49ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:57.048598 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82590d98-8d75-4aa0-a752-3c5080b49ba4 None admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:57.049319 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82590d98-8d75-4aa0-a752-3c5080b49ba4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:57.049625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82590d98-8d75-4aa0-a752-3c5080b49ba4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:57.051069 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82590d98-8d75-4aa0-a752-3c5080b49ba4 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:57.063089 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 640/1279] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:57.068705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278e47d-ef3d-484b-9812-ae01c01072a1 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:57.099853 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:57.166232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278e47d-ef3d-484b-9812-ae01c01072a1 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:57.200231 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb5bd3-96da-46d4-b547-d92ac4b0e529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:57.201024 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb5bd3-96da-46d4-b547-d92ac4b0e529 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:57.201306 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb5bd3-96da-46d4-b547-d92ac4b0e529 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:57.201606 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cb5bd3-96da-46d4-b547-d92ac4b0e529 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:57.208569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cb5bd3-96da-46d4-b547-d92ac4b0e529 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:57.210010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cb5bd3-96da-46d4-b547-d92ac4b0e529 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:57.224837 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-38cb5bd3-96da-46d4-b547-d92ac4b0e529 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 03:18:57.224837 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:57.224837 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:57.224837 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:57.224837 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:57.224837 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:57.224837 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:57.224837 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:57.224837 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:57.224837 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:57.224837 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:57.224837 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:57.224837 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:18:57.224837 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:18:57.224837 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:18:57.224837 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:18:57.229870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:57.229870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:57.229870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:18:57.229870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:18:57.229870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:18:57.229870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:18:57.229870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:57.229870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:57.229870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:57.229870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:57.229870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:57.229870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:18:57.229870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:57.229870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:57.229870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:57.229870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:57.229870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:57.229870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:57.231039 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:57.231039 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:57.231039 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:18:57.231039 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:18:57.231039 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:18:57.231039 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:18:57.231039 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:18:57.231039 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:18:57.231039 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 03:18:57.231039 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:18:57.231039 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 640/1280] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:18:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:57.231931 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82590d98-8d75-4aa0-a752-3c5080b49ba4 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:57.236245 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:57.331288 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82590d98-8d75-4aa0-a752-3c5080b49ba4 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:57.365089 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30626852-9886-4ed5-9a0b-a8fa17b4d946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:57.365745 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30626852-9886-4ed5-9a0b-a8fa17b4d946 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:57.366024 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30626852-9886-4ed5-9a0b-a8fa17b4d946 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:57.366301 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30626852-9886-4ed5-9a0b-a8fa17b4d946 None admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:57.367163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30626852-9886-4ed5-9a0b-a8fa17b4d946 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:57.367468 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30626852-9886-4ed5-9a0b-a8fa17b4d946 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:57.368886 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30626852-9886-4ed5-9a0b-a8fa17b4d946 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:57.385699 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 641/1281] 10.222.0.24 () {56 vars in 1143 bytes} [Fri May 27 03:18:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:18:57.391397 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb5bd3-96da-46d4-b547-d92ac4b0e529 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:57.478589 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:57.487593 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cb5bd3-96da-46d4-b547-d92ac4b0e529 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:57.520704 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827be8a2-8a06-49eb-baa1-28ec0b04948a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:57.521360 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827be8a2-8a06-49eb-baa1-28ec0b04948a None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:57.521625 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827be8a2-8a06-49eb-baa1-28ec0b04948a None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:57.521896 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827be8a2-8a06-49eb-baa1-28ec0b04948a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:57.588356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827be8a2-8a06-49eb-baa1-28ec0b04948a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:57.589734 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827be8a2-8a06-49eb-baa1-28ec0b04948a None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:57.673499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:57.683212 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 641/1282] 10.222.0.24 () {58 vars in 1282 bytes} [Fri May 27 03:18:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:18:57.689098 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30626852-9886-4ed5-9a0b-a8fa17b4d946 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:57.749049 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:57.786240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30626852-9886-4ed5-9a0b-a8fa17b4d946 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:57.819077 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ff6fa-cc0e-49d2-b15d-394dfb8bbea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:57.819853 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ff6fa-cc0e-49d2-b15d-394dfb8bbea7 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:57.820131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ff6fa-cc0e-49d2-b15d-394dfb8bbea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:57.820398 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967ff6fa-cc0e-49d2-b15d-394dfb8bbea7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:57.830982 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967ff6fa-cc0e-49d2-b15d-394dfb8bbea7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:57.832962 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967ff6fa-cc0e-49d2-b15d-394dfb8bbea7 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:57.850790 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-967ff6fa-cc0e-49d2-b15d-394dfb8bbea7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:18:57.850790 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:57.850790 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:57.850790 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:57.850790 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:57.850790 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:57.850790 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:57.850790 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:57.850790 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:57.850790 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:57.850790 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:57.850790 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:57.850790 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:18:57.850790 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:18:57.850790 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:18:57.851860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:18:57.851860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:57.851860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:57.851860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:57.851860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:57.851860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:57.851860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:57.851860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:57.851860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:18:57.851860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:57.851860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:57.851860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:57.851860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:57.851860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:57.851860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:57.851860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:57.851860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:57.852956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:18:57.852956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:18:57.852956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:18:57.852956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:18:57.852956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 642/1283] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:18:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:57.857643 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827be8a2-8a06-49eb-baa1-28ec0b04948a None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:57.954371 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827be8a2-8a06-49eb-baa1-28ec0b04948a None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:57.987218 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a6cb2-a66e-489a-9d05-71e5f1d3bb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:57.987872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a6cb2-a66e-489a-9d05-71e5f1d3bb53 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:57.988154 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a6cb2-a66e-489a-9d05-71e5f1d3bb53 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:57.988430 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a6cb2-a66e-489a-9d05-71e5f1d3bb53 None admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:57.989144 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a6cb2-a66e-489a-9d05-71e5f1d3bb53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:57.989451 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a6cb2-a66e-489a-9d05-71e5f1d3bb53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:57.990824 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a6cb2-a66e-489a-9d05-71e5f1d3bb53 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:58.001654 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 642/1284] 10.222.0.24 () {56 vars in 1109 bytes} [Fri May 27 03:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:58.007236 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ff6fa-cc0e-49d2-b15d-394dfb8bbea7 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:58.103936 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967ff6fa-cc0e-49d2-b15d-394dfb8bbea7 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:58.117299 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:58.136661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a3438-f8d0-41f0-9f5a-42e77568c16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:58.141353 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a3438-f8d0-41f0-9f5a-42e77568c16d None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:58.141353 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a3438-f8d0-41f0-9f5a-42e77568c16d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:58.141353 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a3438-f8d0-41f0-9f5a-42e77568c16d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:58.144936 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a3438-f8d0-41f0-9f5a-42e77568c16d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:58.146363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a3438-f8d0-41f0-9f5a-42e77568c16d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:58.160050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application [None req-a20a3438-f8d0-41f0-9f5a-42e77568c16d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:18:58.160050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:18:58.160050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:18:58.160050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:18:58.160050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:18:58.160050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:18:58.160050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:18:58.160050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:18:58.160050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:18:58.160050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:18:58.160050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:18:58.160050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:18:58.160050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:18:58.160050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:18:58.160050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:18:58.161097 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:18:58.161097 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:18:58.161097 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:18:58.161097 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:18:58.161097 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:18:58.161097 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:18:58.161097 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:18:58.161097 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:18:58.161097 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application with Lock( May 27 03:18:58.161097 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:18:58.161097 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._enter() May 27 03:18:58.161097 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:18:58.161097 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:18:58.161097 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:18:58.161097 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.creator() May 27 03:18:58.161097 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:18:58.161097 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application created_value = creator( May 27 03:18:58.162030 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:18:58.162030 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:18:58.162030 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:18:58.162030 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:18:58.162030 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:18:58.162030 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:18:58.162030 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:18:58.162030 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: ERROR keystone.server.flask.application May 27 03:18:58.162030 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 643/1285] 10.222.0.24 () {56 vars in 1111 bytes} [Fri May 27 03:18:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:18:58.167039 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a6cb2-a66e-489a-9d05-71e5f1d3bb53 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:58.215974 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:18:58.263477 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a6cb2-a66e-489a-9d05-71e5f1d3bb53 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:58.294996 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b898c-c829-49b0-957a-a1baa1a234d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:58.296154 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b898c-c829-49b0-957a-a1baa1a234d0 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:58.296432 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b898c-c829-49b0-957a-a1baa1a234d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:58.296714 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b898c-c829-49b0-957a-a1baa1a234d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:58.297483 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b898c-c829-49b0-957a-a1baa1a234d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:58.297791 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b898c-c829-49b0-957a-a1baa1a234d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:58.299274 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b898c-c829-49b0-957a-a1baa1a234d0 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:58.310852 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 643/1286] 10.222.0.24 () {56 vars in 1148 bytes} [Fri May 27 03:18:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:18:58.317269 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a3438-f8d0-41f0-9f5a-42e77568c16d None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:18:58.413246 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a3438-f8d0-41f0-9f5a-42e77568c16d None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:18:58.446801 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051dd009-9f70-4bb0-8692-09b36aa8c0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:18:58.447462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051dd009-9f70-4bb0-8692-09b36aa8c0c6 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:18:58.447743 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051dd009-9f70-4bb0-8692-09b36aa8c0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:18:58.448016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051dd009-9f70-4bb0-8692-09b36aa8c0c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:18:58.448830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051dd009-9f70-4bb0-8692-09b36aa8c0c6 None admin] RBAC: Adding query filter params (role.id=e5a908bd3ed5445b82a6053246533b88, scope.project.id=6ceffdd955324aec84ba048d2137ae00, user.id=95f2d0aceef4451a83ee89fac5130a0c) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:18:58.449141 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051dd009-9f70-4bb0-8692-09b36aa8c0c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:18:58.450812 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051dd009-9f70-4bb0-8692-09b36aa8c0c6 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:18:58.456611 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 644/1287] 10.222.0.24 () {56 vars in 1374 bytes} [Fri May 27 03:18:58 2022] GET /identity/v3/role_assignments?user.id=95f2d0aceef4451a83ee89fac5130a0c&scope.project.id=6ceffdd955324aec84ba048d2137ae00&role.id=e5a908bd3ed5445b82a6053246533b88 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:19:00.669976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9fe0c-36b8-4de7-a032-9380481d465e None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:00.671092 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9fe0c-36b8-4de7-a032-9380481d465e None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:00.671092 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9fe0c-36b8-4de7-a032-9380481d465e None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:00.672464 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 644/1288] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:19:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:19:00.678442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec999b38-739c-49d5-b893-855826387098 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:00.679035 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec999b38-739c-49d5-b893-855826387098 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:00.679154 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec999b38-739c-49d5-b893-855826387098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:00.697040 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-ec999b38-739c-49d5-b893-855826387098 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:19:00.698257 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ec999b38-739c-49d5-b893-855826387098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:19:00.730620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:19:00.770596 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 645/1289] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:19:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:19:00.792628 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a9fe0c-36b8-4de7-a032-9380481d465e None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:19:00.898979 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a9fe0c-36b8-4de7-a032-9380481d465e None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:19:00.910382 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:19:00.934303 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b2b09-d92b-49d4-9e7b-e2c2f13e99c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:19:00.935064 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b2b09-d92b-49d4-9e7b-e2c2f13e99c3 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:00.935348 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b2b09-d92b-49d4-9e7b-e2c2f13e99c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:00.935629 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b2b09-d92b-49d4-9e7b-e2c2f13e99c3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:00.936309 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b2b09-d92b-49d4-9e7b-e2c2f13e99c3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:19:00.937669 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b2b09-d92b-49d4-9e7b-e2c2f13e99c3 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:19:00.950410 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-430b2b09-d92b-49d4-9e7b-e2c2f13e99c3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:19:00.950410 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:19:00.950410 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:19:00.950410 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:19:00.950410 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:19:00.950410 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:19:00.950410 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:19:00.950410 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:19:00.950410 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:19:00.950410 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:19:00.950410 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:19:00.950410 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:19:00.950410 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:19:00.950410 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:19:00.950410 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:19:00.950410 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:19:00.950410 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:19:00.951732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:19:00.951732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:19:00.951732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:19:00.951732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:19:00.951732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:19:00.951732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:19:00.951732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:19:00.951732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:19:00.951732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:19:00.951732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:19:00.951732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:19:00.951732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:19:00.951732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:19:00.951732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:19:00.951732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:19:00.951732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:19:00.952758 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:19:00.952758 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:19:00.952758 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:19:00.952758 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:19:00.952758 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:19:00.952758 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:19:00.952758 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:19:00.952758 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:19:00.952758 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:19:00.952758 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:19:00.952758 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:19:00.952758 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:19:00.952758 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:19:00.952758 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:19:00.952758 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 645/1290] 10.222.0.24 () {56 vars in 1117 bytes} [Fri May 27 03:19:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:19:00.957765 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec999b38-739c-49d5-b893-855826387098 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:19:01.057441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec999b38-739c-49d5-b893-855826387098 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:19:01.091539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a6340-fcb0-428d-8d60-7287ead7777b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:19:01.092708 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a6340-fcb0-428d-8d60-7287ead7777b None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:01.092987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a6340-fcb0-428d-8d60-7287ead7777b None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:01.093256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a6340-fcb0-428d-8d60-7287ead7777b None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:01.093957 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a6340-fcb0-428d-8d60-7287ead7777b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:19:01.094260 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a6340-fcb0-428d-8d60-7287ead7777b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:19:01.095696 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a6340-fcb0-428d-8d60-7287ead7777b None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:19:01.100405 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 646/1291] 10.222.0.24 () {56 vars in 1116 bytes} [Fri May 27 03:19:00 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:19:01.106071 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b2b09-d92b-49d4-9e7b-e2c2f13e99c3 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:19:01.202580 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b2b09-d92b-49d4-9e7b-e2c2f13e99c3 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:19:01.234523 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658cdec-72f0-4e08-befb-c87c579429e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:19:01.235200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658cdec-72f0-4e08-befb-c87c579429e8 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:01.235483 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658cdec-72f0-4e08-befb-c87c579429e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:01.235757 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658cdec-72f0-4e08-befb-c87c579429e8 None admin] PATH_INFO: `/v3/services` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:01.236457 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658cdec-72f0-4e08-befb-c87c579429e8 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:19:01.236762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658cdec-72f0-4e08-befb-c87c579429e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:19:01.238118 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658cdec-72f0-4e08-befb-c87c579429e8 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:19:01.242426 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 646/1292] 10.222.0.24 () {56 vars in 1116 bytes} [Fri May 27 03:19:01 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:19:03.410566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f6415-c37b-4781-a185-73e295907156 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:03.411093 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f6415-c37b-4781-a185-73e295907156 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:03.411093 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f6415-c37b-4781-a185-73e295907156 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:03.412428 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 647/1293] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:19:03.417517 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604223eb-20f6-4f20-bdef-5b2a6e617f13 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:03.417889 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604223eb-20f6-4f20-bdef-5b2a6e617f13 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:03.418158 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604223eb-20f6-4f20-bdef-5b2a6e617f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:03.435065 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-604223eb-20f6-4f20-bdef-5b2a6e617f13 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:19:03.436834 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-604223eb-20f6-4f20-bdef-5b2a6e617f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:19:03.465856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:19:03.501959 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 647/1294] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:19:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:19:03.524803 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f6415-c37b-4781-a185-73e295907156 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:19:03.555863 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:19:03.630166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f6415-c37b-4781-a185-73e295907156 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:19:03.669002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f8f84-25e4-4fdf-9852-abf36ec3801d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:19:03.669592 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f8f84-25e4-4fdf-9852-abf36ec3801d None admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:03.669875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f8f84-25e4-4fdf-9852-abf36ec3801d None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:03.670143 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f8f84-25e4-4fdf-9852-abf36ec3801d None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:03.670883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f8f84-25e4-4fdf-9852-abf36ec3801d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:19:03.672248 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f8f84-25e4-4fdf-9852-abf36ec3801d None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:19:03.766571 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 648/1295] 10.222.0.24 () {60 vars in 1141 bytes} [Fri May 27 03:19:03 2022] POST /identity/v3/services => generated 246 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:19:05.962615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970fbd9-d850-4cb7-ac60-fa810312dbab None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:05.963220 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970fbd9-d850-4cb7-ac60-fa810312dbab None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:05.963220 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970fbd9-d850-4cb7-ac60-fa810312dbab None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:05.964559 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 648/1296] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:19:05.970655 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140d59f-a6b9-44cb-9272-f278d7f114b8 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:05.971125 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140d59f-a6b9-44cb-9272-f278d7f114b8 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:05.971226 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140d59f-a6b9-44cb-9272-f278d7f114b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:05.997153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-9140d59f-a6b9-44cb-9272-f278d7f114b8 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:19:05.998390 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-9140d59f-a6b9-44cb-9272-f278d7f114b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:19:06.070401 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 649/1297] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:19:05 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:19:06.094706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8970fbd9-d850-4cb7-ac60-fa810312dbab None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:19:06.191192 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8970fbd9-d850-4cb7-ac60-fa810312dbab None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:19:06.224465 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b6969-75ad-4f74-8c00-e9b15428ffae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:19:06.225183 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b6969-75ad-4f74-8c00-e9b15428ffae None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:06.225464 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b6969-75ad-4f74-8c00-e9b15428ffae None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:06.225733 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b6969-75ad-4f74-8c00-e9b15428ffae None admin] PATH_INFO: `/v3/services/placement` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:06.226377 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b6969-75ad-4f74-8c00-e9b15428ffae None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:19:06.227746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b6969-75ad-4f74-8c00-e9b15428ffae None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:19:06.239200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-960b6969-75ad-4f74-8c00-e9b15428ffae None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:19:06.239200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:19:06.239200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:19:06.239200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:19:06.239200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:19:06.239200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:19:06.239200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:19:06.239200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:19:06.239200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:19:06.239200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:19:06.239200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:19:06.239200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:19:06.239200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:19:06.239200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:19:06.239200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:19:06.239200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:19:06.239200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:19:06.240386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:19:06.240386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:19:06.240386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:19:06.240386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:19:06.240386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:19:06.240386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:19:06.240386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:19:06.240386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:19:06.240386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:19:06.240386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:19:06.240386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:19:06.240386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:19:06.240386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:19:06.240386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:19:06.240386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:19:06.240386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:19:06.241678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:19:06.241678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:19:06.241678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:19:06.241678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:19:06.241678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:19:06.241678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:19:06.241678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:19:06.241678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:19:06.241678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:19:06.241678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:19:06.241678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:19:06.241678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:19:06.241678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:19:06.241678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:19:06.241678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 649/1298] 10.222.0.24 () {56 vars in 1117 bytes} [Fri May 27 03:19:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:19:06.246411 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140d59f-a6b9-44cb-9272-f278d7f114b8 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:19:06.265357 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:19:06.347743 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140d59f-a6b9-44cb-9272-f278d7f114b8 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:19:06.382600 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d9a20-7fe1-4a5c-8df3-2942341caaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:19:06.383286 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d9a20-7fe1-4a5c-8df3-2942341caaf6 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:06.383606 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d9a20-7fe1-4a5c-8df3-2942341caaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:06.383878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d9a20-7fe1-4a5c-8df3-2942341caaf6 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:06.384577 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d9a20-7fe1-4a5c-8df3-2942341caaf6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:19:06.384882 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d9a20-7fe1-4a5c-8df3-2942341caaf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:19:06.386238 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d9a20-7fe1-4a5c-8df3-2942341caaf6 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:19:06.392001 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 650/1299] 10.222.0.24 () {56 vars in 1116 bytes} [Fri May 27 03:19:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:19:06.397978 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b6969-75ad-4f74-8c00-e9b15428ffae None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:19:06.428418 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:19:06.493749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b6969-75ad-4f74-8c00-e9b15428ffae None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:19:06.527999 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8804fa4-142c-441e-b24d-121b7c7df9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:19:06.528645 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8804fa4-142c-441e-b24d-121b7c7df9c7 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:06.528925 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8804fa4-142c-441e-b24d-121b7c7df9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:06.529200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8804fa4-142c-441e-b24d-121b7c7df9c7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:06.530584 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8804fa4-142c-441e-b24d-121b7c7df9c7 None admin] RBAC: Adding query filter params (interface=public, service_id=a453bb8af694431d8fda4d33ae29f5cd, region_id=RegionOne) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:19:06.530924 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8804fa4-142c-441e-b24d-121b7c7df9c7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:19:06.532282 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8804fa4-142c-441e-b24d-121b7c7df9c7 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:19:06.539472 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 650/1300] 10.222.0.24 () {56 vars in 1251 bytes} [Fri May 27 03:19:06 2022] GET /identity/v3/endpoints?service_id=a453bb8af694431d8fda4d33ae29f5cd&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:19:06.545029 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d9a20-7fe1-4a5c-8df3-2942341caaf6 None admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:19:06.624311 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:19:06.642621 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d9a20-7fe1-4a5c-8df3-2942341caaf6 None admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:19:06.679244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6938a-f0a9-46de-a992-a116330b9d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:19:06.679893 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6938a-f0a9-46de-a992-a116330b9d48 None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:06.680174 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6938a-f0a9-46de-a992-a116330b9d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:06.680449 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6938a-f0a9-46de-a992-a116330b9d48 None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:06.681968 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6938a-f0a9-46de-a992-a116330b9d48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:19:06.683366 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6938a-f0a9-46de-a992-a116330b9d48 None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:19:06.690045 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 651/1301] 10.222.0.24 () {56 vars in 1087 bytes} [Fri May 27 03:19:06 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:19:08.868308 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc61b6-ec8b-4c74-844a-d12b62d6d16a None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:08.868843 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc61b6-ec8b-4c74-844a-d12b62d6d16a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:08.868843 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc61b6-ec8b-4c74-844a-d12b62d6d16a None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:08.870207 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 651/1302] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:19:08.876062 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f6155-c769-4784-a608-a41591df6179 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:08.876511 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f6155-c769-4784-a608-a41591df6179 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:08.876603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41f6155-c769-4784-a608-a41591df6179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:08.893110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-f41f6155-c769-4784-a608-a41591df6179 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:19:08.894287 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f41f6155-c769-4784-a608-a41591df6179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:19:08.960177 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 652/1303] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:19:08 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:19:08.982155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc61b6-ec8b-4c74-844a-d12b62d6d16a None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:19:09.073628 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:19:09.081060 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc61b6-ec8b-4c74-844a-d12b62d6d16a None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:19:09.114788 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa58e73-0bb7-48ec-83ef-4ff826c0d673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:19:09.115608 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa58e73-0bb7-48ec-83ef-4ff826c0d673 None admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:09.115885 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa58e73-0bb7-48ec-83ef-4ff826c0d673 None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:09.116171 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa58e73-0bb7-48ec-83ef-4ff826c0d673 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:09.116807 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa58e73-0bb7-48ec-83ef-4ff826c0d673 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:19:09.118163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa58e73-0bb7-48ec-83ef-4ff826c0d673 None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:19:09.130366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application [None req-ffa58e73-0bb7-48ec-83ef-4ff826c0d673 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:19:09.130366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:19:09.130366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:19:09.130366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:19:09.130366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:19:09.130366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:19:09.130366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:19:09.130366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:19:09.130366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:19:09.130366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:19:09.130366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:19:09.130366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:19:09.130366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:19:09.130366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:19:09.130366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:19:09.130366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:19:09.130366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:19:09.131569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:19:09.131569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:19:09.131569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:19:09.131569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:19:09.131569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:19:09.131569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:19:09.131569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:19:09.131569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:19:09.131569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:19:09.131569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:19:09.131569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:19:09.131569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:19:09.131569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:19:09.131569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:19:09.131569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:19:09.131569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:19:09.132595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:19:09.132595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application with Lock( May 27 03:19:09.132595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:19:09.132595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self._enter() May 27 03:19:09.132595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:19:09.132595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:19:09.132595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:19:09.132595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application return self.creator() May 27 03:19:09.132595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:19:09.132595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application created_value = creator( May 27 03:19:09.132595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:19:09.132595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:19:09.132595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:19:09.132595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: ERROR keystone.server.flask.application May 27 03:19:09.132595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 652/1304] 10.222.0.24 () {56 vars in 1117 bytes} [Fri May 27 03:19:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:19:09.137591 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f6155-c769-4784-a608-a41591df6179 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:19:09.210985 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:19:09.233915 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41f6155-c769-4784-a608-a41591df6179 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:19:09.266131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca1627-c741-4ef1-9697-f8bdd97db8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:19:09.266793 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca1627-c741-4ef1-9697-f8bdd97db8be None admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:09.267091 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca1627-c741-4ef1-9697-f8bdd97db8be None admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:09.267367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca1627-c741-4ef1-9697-f8bdd97db8be None admin] PATH_INFO: `/v3/services` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:09.268067 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca1627-c741-4ef1-9697-f8bdd97db8be None admin] RBAC: Adding query filter params (name=placement) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:19:09.268370 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca1627-c741-4ef1-9697-f8bdd97db8be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:19:09.269749 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ca1627-c741-4ef1-9697-f8bdd97db8be None admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:19:09.274943 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 653/1305] 10.222.0.24 () {56 vars in 1116 bytes} [Fri May 27 03:19:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:19:09.281242 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa58e73-0bb7-48ec-83ef-4ff826c0d673 None admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:19:09.377232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa58e73-0bb7-48ec-83ef-4ff826c0d673 None admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:19:09.409291 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def1eab-ac0b-466a-b366-c4944f7e0a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:19:09.409879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def1eab-ac0b-466a-b366-c4944f7e0a6f None admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:09.410163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def1eab-ac0b-466a-b366-c4944f7e0a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:09.410439 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def1eab-ac0b-466a-b366-c4944f7e0a6f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:09.411197 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def1eab-ac0b-466a-b366-c4944f7e0a6f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:19:09.412560 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def1eab-ac0b-466a-b366-c4944f7e0a6f None admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:19:09.537336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 653/1306] 10.222.0.24 () {60 vars in 1144 bytes} [Fri May 27 03:19:09 2022] POST /identity/v3/endpoints => generated 331 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:19:19.271863 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=30 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:19:19.532939 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:17.481664 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60cfa03-6a3b-421e-88fe-8d0d0b11a274 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:17.482687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60cfa03-6a3b-421e-88fe-8d0d0b11a274 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:17.483216 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60cfa03-6a3b-421e-88fe-8d0d0b11a274 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:17.489957 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0d272-72c4-4b73-b3b6-b73cfe40bd6b None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:17.490790 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0d272-72c4-4b73-b3b6-b73cfe40bd6b None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:17.491329 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0d272-72c4-4b73-b3b6-b73cfe40bd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:17.495447 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 654/1307] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:26:17 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:26:17.566431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-31b0d272-72c4-4b73-b3b6-b73cfe40bd6b None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:17.569224 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-31b0d272-72c4-4b73-b3b6-b73cfe40bd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:17.708223 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 654/1308] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:26:17.743650 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2694d25-f6c9-47f4-af3a-974394901698 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:17.744985 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2694d25-f6c9-47f4-af3a-974394901698 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:17.745491 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2694d25-f6c9-47f4-af3a-974394901698 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:17.747499 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 655/1309] 10.222.0.24 () {54 vars in 964 bytes} [Fri May 27 03:26:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:26:17.756542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cda255-c975-4b70-8d85-f951396e44ef None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:17.756973 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cda255-c975-4b70-8d85-f951396e44ef None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:17.757255 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cda255-c975-4b70-8d85-f951396e44ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:17.884567 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-a1cda255-c975-4b70-8d85-f951396e44ef None None] MFA Rules not processed for user `1af0160cd3da4a0bb391b81f083715d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:17.886730 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a1cda255-c975-4b70-8d85-f951396e44ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:17.925271 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:18.038968 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2694d25-f6c9-47f4-af3a-974394901698 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:18.041895 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a2694d25-f6c9-47f4-af3a-974394901698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:18.044325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 655/1310] 10.222.0.24 () {58 vars in 1061 bytes} [Fri May 27 03:26:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:18.180490 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2694d25-f6c9-47f4-af3a-974394901698 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:18.262189 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cf665d-a7b6-47fa-a3ca-9e8b77367d9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:18.263020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf665d-a7b6-47fa-a3ca-9e8b77367d9a service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:18.263372 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf665d-a7b6-47fa-a3ca-9e8b77367d9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:18.263721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf665d-a7b6-47fa-a3ca-9e8b77367d9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:18.265495 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f1cf665d-a7b6-47fa-a3ca-9e8b77367d9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:18.323332 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:18.386242 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf665d-a7b6-47fa-a3ca-9e8b77367d9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:18.387822 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cf665d-a7b6-47fa-a3ca-9e8b77367d9a service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:18.407075 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 656/1311] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:26:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:26:20.863445 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d35a3-8fa5-4292-814d-0d697438db3e None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:20.863924 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d35a3-8fa5-4292-814d-0d697438db3e None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:20.864029 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d35a3-8fa5-4292-814d-0d697438db3e None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:20.865349 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 656/1312] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:26:20.873059 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa9923-89cb-4d54-904d-97fe873ca6bc None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:20.873529 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa9923-89cb-4d54-904d-97fe873ca6bc None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:20.873624 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa9923-89cb-4d54-904d-97fe873ca6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:20.973461 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-dbaa9923-89cb-4d54-904d-97fe873ca6bc None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:20.974680 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-dbaa9923-89cb-4d54-904d-97fe873ca6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:21.079341 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 657/1313] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:21.116095 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d35a3-8fa5-4292-814d-0d697438db3e None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:21.117756 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-999d35a3-8fa5-4292-814d-0d697438db3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:21.227479 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d35a3-8fa5-4292-814d-0d697438db3e None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:21.304236 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:21.307857 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e58bb1-dfee-4391-8781-fddce9643f21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:21.308508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e58bb1-dfee-4391-8781-fddce9643f21 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:21.308877 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e58bb1-dfee-4391-8781-fddce9643f21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:21.309155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e58bb1-dfee-4391-8781-fddce9643f21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:21.310810 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-18e58bb1-dfee-4391-8781-fddce9643f21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:21.426450 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e58bb1-dfee-4391-8781-fddce9643f21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:21.428066 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e58bb1-dfee-4391-8781-fddce9643f21 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:21.450206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 657/1314] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:26:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:26:24.299854 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa380e3a-5d5d-498e-a926-f18d20c79f9b None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:24.300385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa380e3a-5d5d-498e-a926-f18d20c79f9b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:24.300385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa380e3a-5d5d-498e-a926-f18d20c79f9b None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:24.301779 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 658/1315] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:26:24.306978 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475afba-0ddc-46c8-b549-1bbe56124f48 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:24.307361 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475afba-0ddc-46c8-b549-1bbe56124f48 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:24.307522 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475afba-0ddc-46c8-b549-1bbe56124f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:24.369737 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-4475afba-0ddc-46c8-b549-1bbe56124f48 None None] MFA Rules not processed for user `3e25056b9cac4b5cb4ff28c737a1cbdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:24.370918 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-4475afba-0ddc-46c8-b549-1bbe56124f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:24.419441 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:24.462380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 658/1316] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:26:24 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:24.501260 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa380e3a-5d5d-498e-a926-f18d20c79f9b None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:24.502794 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-aa380e3a-5d5d-498e-a926-f18d20c79f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:24.513056 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:24.607674 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa380e3a-5d5d-498e-a926-f18d20c79f9b None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:24.679616 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de050983-cdc7-4f20-9fed-a31752ecfcc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:24.680190 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de050983-cdc7-4f20-9fed-a31752ecfcc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:24.680474 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de050983-cdc7-4f20-9fed-a31752ecfcc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:24.680747 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de050983-cdc7-4f20-9fed-a31752ecfcc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:24.682965 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-de050983-cdc7-4f20-9fed-a31752ecfcc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:24.743491 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:24.788792 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de050983-cdc7-4f20-9fed-a31752ecfcc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:24.790217 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de050983-cdc7-4f20-9fed-a31752ecfcc8 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:24.809179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 659/1317] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:26:24 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:26:27.877110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21293e9f-7649-4146-bc24-643d7f22bfce None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:27.878959 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21293e9f-7649-4146-bc24-643d7f22bfce None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:27.878959 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21293e9f-7649-4146-bc24-643d7f22bfce None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:27.879153 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 659/1318] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:26:27.884947 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23ee2a-0e1f-4c05-a323-21ff5419fd77 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:27.885410 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23ee2a-0e1f-4c05-a323-21ff5419fd77 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:27.885509 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23ee2a-0e1f-4c05-a323-21ff5419fd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:27.950625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-ec23ee2a-0e1f-4c05-a323-21ff5419fd77 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:27.951854 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ec23ee2a-0e1f-4c05-a323-21ff5419fd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:28.046219 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 660/1319] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:26:27 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:26:28.100252 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e6620-31df-4d92-a5e7-8037fd9f6588 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:28.100542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e6620-31df-4d92-a5e7-8037fd9f6588 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:28.100816 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e6620-31df-4d92-a5e7-8037fd9f6588 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:28.102292 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 660/1320] 10.222.0.24 () {54 vars in 964 bytes} [Fri May 27 03:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:26:28.108064 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665516e-6966-4dc5-8154-912d30eafe0a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:28.108358 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665516e-6966-4dc5-8154-912d30eafe0a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:28.108637 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665516e-6966-4dc5-8154-912d30eafe0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:28.129340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:28.182830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-1665516e-6966-4dc5-8154-912d30eafe0a None None] MFA Rules not processed for user `1af0160cd3da4a0bb391b81f083715d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:28.184071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1665516e-6966-4dc5-8154-912d30eafe0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:28.258259 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 661/1321] 10.222.0.24 () {58 vars in 1061 bytes} [Fri May 27 03:26:28 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:28.268162 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e6620-31df-4d92-a5e7-8037fd9f6588 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:28.269699 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-656e6620-31df-4d92-a5e7-8037fd9f6588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:28.370983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e6620-31df-4d92-a5e7-8037fd9f6588 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:28.441880 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc3fdc-2118-4f15-b7dc-83a54636fb6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:28.442478 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc3fdc-2118-4f15-b7dc-83a54636fb6a service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:28.442755 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc3fdc-2118-4f15-b7dc-83a54636fb6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:28.443090 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc3fdc-2118-4f15-b7dc-83a54636fb6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:28.444753 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-cffc3fdc-2118-4f15-b7dc-83a54636fb6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:28.452608 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:28.554923 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc3fdc-2118-4f15-b7dc-83a54636fb6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:28.556994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc3fdc-2118-4f15-b7dc-83a54636fb6a service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:28.576047 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 661/1322] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:26:28 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:26:32.380926 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8aa4a-a0a4-4d2a-b780-65f1ff77594b None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:32.381449 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8aa4a-a0a4-4d2a-b780-65f1ff77594b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:32.381449 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8aa4a-a0a4-4d2a-b780-65f1ff77594b None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:32.382749 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 662/1323] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:26:32.388123 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615b839-6fcb-4279-b82c-e4290006ce9d None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:32.388627 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615b839-6fcb-4279-b82c-e4290006ce9d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:32.388785 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615b839-6fcb-4279-b82c-e4290006ce9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:32.453504 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e615b839-6fcb-4279-b82c-e4290006ce9d None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:32.454987 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e615b839-6fcb-4279-b82c-e4290006ce9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:32.512489 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:32.550579 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 662/1324] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:26:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:32.585847 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8aa4a-a0a4-4d2a-b780-65f1ff77594b None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:32.587969 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-7ca8aa4a-a0a4-4d2a-b780-65f1ff77594b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:32.690516 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8aa4a-a0a4-4d2a-b780-65f1ff77594b None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:32.702353 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:32.763148 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d2566c-1078-4349-b69b-4f6e90c904c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:32.763788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2566c-1078-4349-b69b-4f6e90c904c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:32.764069 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2566c-1078-4349-b69b-4f6e90c904c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:32.764340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2566c-1078-4349-b69b-4f6e90c904c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:32.765961 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b5d2566c-1078-4349-b69b-4f6e90c904c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:32.874619 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d2566c-1078-4349-b69b-4f6e90c904c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:32.876080 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d2566c-1078-4349-b69b-4f6e90c904c9 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:32.894041 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 663/1325] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:26:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:26:35.167950 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dea556-2a6e-4d7d-90f2-b579073d3ac5 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:35.168480 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dea556-2a6e-4d7d-90f2-b579073d3ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:35.168575 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dea556-2a6e-4d7d-90f2-b579073d3ac5 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:35.169891 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 663/1326] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:26:35.175259 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a5e97-117c-4910-8688-1abba49be81c None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:35.175684 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a5e97-117c-4910-8688-1abba49be81c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:35.175803 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a5e97-117c-4910-8688-1abba49be81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:35.224480 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:35.242566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-528a5e97-117c-4910-8688-1abba49be81c None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:35.244491 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-528a5e97-117c-4910-8688-1abba49be81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:35.339926 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 664/1327] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:26:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:35.375377 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dea556-2a6e-4d7d-90f2-b579073d3ac5 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:35.376927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d2dea556-2a6e-4d7d-90f2-b579073d3ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:35.477549 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dea556-2a6e-4d7d-90f2-b579073d3ac5 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:35.552195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21268275-95fe-4357-a995-4b0d0b919ce7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:35.552848 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21268275-95fe-4357-a995-4b0d0b919ce7 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:35.553130 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21268275-95fe-4357-a995-4b0d0b919ce7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:35.553519 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21268275-95fe-4357-a995-4b0d0b919ce7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:35.555345 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-21268275-95fe-4357-a995-4b0d0b919ce7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:35.574980 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:35.668654 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21268275-95fe-4357-a995-4b0d0b919ce7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:35.670108 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21268275-95fe-4357-a995-4b0d0b919ce7 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:35.687880 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 664/1328] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:26:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:26:37.969340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68527892-439d-4a77-b327-4d68ab40f075 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:37.969913 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68527892-439d-4a77-b327-4d68ab40f075 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:37.969913 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68527892-439d-4a77-b327-4d68ab40f075 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:37.971436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 665/1329] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:26:37.976541 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601eaec-1634-4b33-8f2c-ac194d935412 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:37.976967 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601eaec-1634-4b33-8f2c-ac194d935412 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:37.977221 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7601eaec-1634-4b33-8f2c-ac194d935412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:38.044556 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-7601eaec-1634-4b33-8f2c-ac194d935412 None None] MFA Rules not processed for user `3e25056b9cac4b5cb4ff28c737a1cbdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:38.045774 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-7601eaec-1634-4b33-8f2c-ac194d935412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:38.111129 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:38.141994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 665/1330] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:26:37 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:38.179839 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68527892-439d-4a77-b327-4d68ab40f075 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:38.181498 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-68527892-439d-4a77-b327-4d68ab40f075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:38.290974 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68527892-439d-4a77-b327-4d68ab40f075 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:38.310747 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:38.365390 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac5eb0d-b10c-4597-bb14-af2d17f87377 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:38.366039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5eb0d-b10c-4597-bb14-af2d17f87377 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:38.366316 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5eb0d-b10c-4597-bb14-af2d17f87377 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:38.366589 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5eb0d-b10c-4597-bb14-af2d17f87377 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:38.368271 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3ac5eb0d-b10c-4597-bb14-af2d17f87377 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:38.474136 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac5eb0d-b10c-4597-bb14-af2d17f87377 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:38.475607 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac5eb0d-b10c-4597-bb14-af2d17f87377 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:38.494235 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 666/1331] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:26:38 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:26:41.166534 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2c037-cda0-45ce-870d-471f8a1b7009 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:41.167076 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2c037-cda0-45ce-870d-471f8a1b7009 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:41.167076 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2c037-cda0-45ce-870d-471f8a1b7009 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:41.168425 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 666/1332] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:26:41.173550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16db86-0163-4ee0-aa1e-c5fd0105f729 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:41.173968 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16db86-0163-4ee0-aa1e-c5fd0105f729 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:41.174108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec16db86-0163-4ee0-aa1e-c5fd0105f729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:41.238950 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:41.241726 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-ec16db86-0163-4ee0-aa1e-c5fd0105f729 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:41.243039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ec16db86-0163-4ee0-aa1e-c5fd0105f729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:41.339776 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 667/1333] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:26:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:41.381672 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2c037-cda0-45ce-870d-471f8a1b7009 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:41.383554 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-56a2c037-cda0-45ce-870d-471f8a1b7009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:41.501485 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2c037-cda0-45ce-870d-471f8a1b7009 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:41.583515 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738094c-81aa-4fd7-9b57-7ae7318398aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:41.584271 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738094c-81aa-4fd7-9b57-7ae7318398aa service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:41.584619 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738094c-81aa-4fd7-9b57-7ae7318398aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:41.584991 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738094c-81aa-4fd7-9b57-7ae7318398aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:41.586757 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d738094c-81aa-4fd7-9b57-7ae7318398aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:41.618296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:41.720936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738094c-81aa-4fd7-9b57-7ae7318398aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:41.722400 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738094c-81aa-4fd7-9b57-7ae7318398aa service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:41.741640 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 667/1334] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:26:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:26:43.991626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3c4ef-2ee5-4d0e-8a50-cf82050a0608 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:43.992183 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3c4ef-2ee5-4d0e-8a50-cf82050a0608 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:43.992183 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3c4ef-2ee5-4d0e-8a50-cf82050a0608 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:43.994077 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 668/1335] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:26:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:26:43.999359 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b72808-d03e-4700-b2a3-e96718060bbd None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:43.999742 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b72808-d03e-4700-b2a3-e96718060bbd None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:43.999905 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b72808-d03e-4700-b2a3-e96718060bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:44.061637 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d3b72808-d03e-4700-b2a3-e96718060bbd None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:44.062805 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d3b72808-d03e-4700-b2a3-e96718060bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:44.136231 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:44.157139 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 668/1336] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:26:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:44.197201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3c4ef-2ee5-4d0e-8a50-cf82050a0608 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:44.198728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8ec3c4ef-2ee5-4d0e-8a50-cf82050a0608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:44.298254 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3c4ef-2ee5-4d0e-8a50-cf82050a0608 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:44.325656 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:44.368910 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1fefee-e649-4423-8431-8c11b620535d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:44.370476 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1fefee-e649-4423-8431-8c11b620535d service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:44.371093 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1fefee-e649-4423-8431-8c11b620535d service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:44.371537 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1fefee-e649-4423-8431-8c11b620535d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:44.373541 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-0a1fefee-e649-4423-8431-8c11b620535d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:44.477708 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1fefee-e649-4423-8431-8c11b620535d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:44.479711 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1fefee-e649-4423-8431-8c11b620535d service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:44.498070 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 669/1337] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:26:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:26:47.392280 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e19037-bbd9-453a-beeb-41bd336575a2 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:47.392280 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e19037-bbd9-453a-beeb-41bd336575a2 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:47.392737 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e19037-bbd9-453a-beeb-41bd336575a2 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:47.393855 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 669/1338] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:26:47.399120 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20c996-685d-4ca8-8960-e8c9d1445734 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:47.399621 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20c996-685d-4ca8-8960-e8c9d1445734 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:47.399724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20c996-685d-4ca8-8960-e8c9d1445734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:47.469783 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-2f20c996-685d-4ca8-8960-e8c9d1445734 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:47.471028 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-2f20c996-685d-4ca8-8960-e8c9d1445734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:47.480017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:47.567474 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 670/1339] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:26:47 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:47.602635 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e19037-bbd9-453a-beeb-41bd336575a2 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:47.606255 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-17e19037-bbd9-453a-beeb-41bd336575a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:47.714990 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e19037-bbd9-453a-beeb-41bd336575a2 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:47.718426 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb80b1-b3d3-4b6e-8e0d-05da0ed2f3b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:47.719100 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb80b1-b3d3-4b6e-8e0d-05da0ed2f3b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:47.719380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb80b1-b3d3-4b6e-8e0d-05da0ed2f3b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:47.719650 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb80b1-b3d3-4b6e-8e0d-05da0ed2f3b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:47.721395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b4cb80b1-b3d3-4b6e-8e0d-05da0ed2f3b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:47.822325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb80b1-b3d3-4b6e-8e0d-05da0ed2f3b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:47.823778 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb80b1-b3d3-4b6e-8e0d-05da0ed2f3b1 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:47.834204 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:47.843412 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 670/1340] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:26:47 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:26:52.418822 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9f74b-6b3c-43d7-bd2d-2ac1a6a12681 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:52.419868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9f74b-6b3c-43d7-bd2d-2ac1a6a12681 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:52.419868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a9f74b-6b3c-43d7-bd2d-2ac1a6a12681 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:52.420787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 671/1341] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:26:52.426085 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deba591-c51a-4516-90a4-7b12f7648771 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:52.426526 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deba591-c51a-4516-90a4-7b12f7648771 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:52.426778 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deba591-c51a-4516-90a4-7b12f7648771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:52.481052 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-1deba591-c51a-4516-90a4-7b12f7648771 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:52.482409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1deba591-c51a-4516-90a4-7b12f7648771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:52.561906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 671/1342] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:26:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:52.600875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a9f74b-6b3c-43d7-bd2d-2ac1a6a12681 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:52.612388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a9f74b-6b3c-43d7-bd2d-2ac1a6a12681 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:52.615845 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a3bad-ddf3-43d6-a30d-311f8ebf4a84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:52.616477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a3bad-ddf3-43d6-a30d-311f8ebf4a84 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:52.616771 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a3bad-ddf3-43d6-a30d-311f8ebf4a84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:52.617029 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a3bad-ddf3-43d6-a30d-311f8ebf4a84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:52.625745 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a3bad-ddf3-43d6-a30d-311f8ebf4a84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:52.627203 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a3bad-ddf3-43d6-a30d-311f8ebf4a84 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:52.637203 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 672/1343] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:26:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:26:55.848617 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fed82-4da0-4930-a4ed-29f600bc73ea None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:55.849168 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fed82-4da0-4930-a4ed-29f600bc73ea None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:55.849252 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fed82-4da0-4930-a4ed-29f600bc73ea None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:55.850566 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 672/1344] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:26:55.856357 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c10901-ada6-4172-836d-f3ed1ee50e00 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:55.856764 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c10901-ada6-4172-836d-f3ed1ee50e00 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:55.856914 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c10901-ada6-4172-836d-f3ed1ee50e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:55.884439 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-31c10901-ada6-4172-836d-f3ed1ee50e00 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:55.885621 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-31c10901-ada6-4172-836d-f3ed1ee50e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:55.893657 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 673/1345] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:26:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:55.930549 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8fed82-4da0-4930-a4ed-29f600bc73ea None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:55.941245 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8fed82-4da0-4930-a4ed-29f600bc73ea None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:55.944706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920e34e-b10b-4f7e-b424-d6980e9064e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:55.945335 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920e34e-b10b-4f7e-b424-d6980e9064e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:55.945612 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920e34e-b10b-4f7e-b424-d6980e9064e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:55.945882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920e34e-b10b-4f7e-b424-d6980e9064e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:55.955626 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920e34e-b10b-4f7e-b424-d6980e9064e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:55.957053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920e34e-b10b-4f7e-b424-d6980e9064e7 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:55.967566 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 673/1346] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:26:55 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:27:00.322545 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9ff28-9a1b-4a87-b6da-2b7718806027 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:00.324801 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9ff28-9a1b-4a87-b6da-2b7718806027 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:00.324801 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9ff28-9a1b-4a87-b6da-2b7718806027 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:00.324801 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 674/1347] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:27:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:00.330280 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e1592-b1c9-40cf-a912-8f6fbc3927d3 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:00.330711 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e1592-b1c9-40cf-a912-8f6fbc3927d3 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:00.330915 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6e1592-b1c9-40cf-a912-8f6fbc3927d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:00.348450 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-8f6e1592-b1c9-40cf-a912-8f6fbc3927d3 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:00.349645 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-8f6e1592-b1c9-40cf-a912-8f6fbc3927d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:00.357484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 674/1348] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:27:00.395797 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9ff28-9a1b-4a87-b6da-2b7718806027 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:00.406690 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9ff28-9a1b-4a87-b6da-2b7718806027 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:00.410681 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c2be31-dca5-4c08-a27e-6164f006d666 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:00.411292 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2be31-dca5-4c08-a27e-6164f006d666 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:00.411569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2be31-dca5-4c08-a27e-6164f006d666 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:00.411864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c2be31-dca5-4c08-a27e-6164f006d666 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:00.420719 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c2be31-dca5-4c08-a27e-6164f006d666 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:00.422189 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c2be31-dca5-4c08-a27e-6164f006d666 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:00.432693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 675/1349] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:27:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:27:05.899532 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f064f69-0004-47e0-8134-a836bdb24813 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:05.900926 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f064f69-0004-47e0-8134-a836bdb24813 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:05.901378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f064f69-0004-47e0-8134-a836bdb24813 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:05.902932 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 675/1350] 10.222.0.24 () {54 vars in 921 bytes} [Fri May 27 03:27:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:05.908034 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd684c78-d250-42ef-834b-f787b4ad3b4e None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:05.908788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd684c78-d250-42ef-834b-f787b4ad3b4e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:05.909220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd684c78-d250-42ef-834b-f787b4ad3b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:05.984731 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-dd684c78-d250-42ef-834b-f787b4ad3b4e None None] MFA Rules not processed for user `95f2d0aceef4451a83ee89fac5130a0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:05.986774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-dd684c78-d250-42ef-834b-f787b4ad3b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:06.050603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:06.094624 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 676/1351] 10.222.0.24 () {58 vars in 1018 bytes} [Fri May 27 03:27:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:10.343778 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=56 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:10.437093 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2489088-6928-432e-bd37-519392210463 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:10.437651 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2489088-6928-432e-bd37-519392210463 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:10.437936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2489088-6928-432e-bd37-519392210463 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:10.439373 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 676/1352] 10.222.0.24 () {54 vars in 921 bytes} [Fri May 27 03:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:10.445078 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bf5d0-daef-4c3d-a882-7c352d0fc32d None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:10.446131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bf5d0-daef-4c3d-a882-7c352d0fc32d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:10.446221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bf5d0-daef-4c3d-a882-7c352d0fc32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:10.465874 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-5e5bf5d0-daef-4c3d-a882-7c352d0fc32d None None] MFA Rules not processed for user `95f2d0aceef4451a83ee89fac5130a0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:10.467198 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5e5bf5d0-daef-4c3d-a882-7c352d0fc32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:10.475992 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 677/1353] 10.222.0.24 () {58 vars in 1018 bytes} [Fri May 27 03:27:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:11.822885 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf5fca-7670-4f8c-8a99-544ea6ff4ac3 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:11.823746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf5fca-7670-4f8c-8a99-544ea6ff4ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:11.824047 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf5fca-7670-4f8c-8a99-544ea6ff4ac3 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:11.825412 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 677/1354] 10.222.0.24 () {54 vars in 921 bytes} [Fri May 27 03:27:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:11.830811 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb0d8b-bd2c-4f6e-8bc5-c38d00ac39c2 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:11.831293 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb0d8b-bd2c-4f6e-8bc5-c38d00ac39c2 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:11.831491 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb0d8b-bd2c-4f6e-8bc5-c38d00ac39c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:11.849527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-9dbb0d8b-bd2c-4f6e-8bc5-c38d00ac39c2 None None] MFA Rules not processed for user `95f2d0aceef4451a83ee89fac5130a0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:11.850728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-9dbb0d8b-bd2c-4f6e-8bc5-c38d00ac39c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:11.862083 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 678/1355] 10.222.0.24 () {58 vars in 1018 bytes} [Fri May 27 03:27:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:15.725404 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e24261-4234-40da-b132-c42ef0b87062 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:15.725921 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e24261-4234-40da-b132-c42ef0b87062 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:15.725921 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e24261-4234-40da-b132-c42ef0b87062 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:15.727263 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 678/1356] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:27:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:15.732581 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d47d9e-4bed-42aa-bbbb-93f6bbb36293 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:15.732993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d47d9e-4bed-42aa-bbbb-93f6bbb36293 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:15.733132 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d47d9e-4bed-42aa-bbbb-93f6bbb36293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:15.751253 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-f0d47d9e-4bed-42aa-bbbb-93f6bbb36293 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:15.755726 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f0d47d9e-4bed-42aa-bbbb-93f6bbb36293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:15.761586 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 679/1357] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:27:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:15.813206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382e515-dd6b-4104-8b6f-1319348f406c None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:15.814286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382e515-dd6b-4104-8b6f-1319348f406c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:15.814731 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382e515-dd6b-4104-8b6f-1319348f406c None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:15.816119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 679/1358] 10.222.0.24 () {54 vars in 961 bytes} [Fri May 27 03:27:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:15.823513 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc99d2b-26b5-48ee-ac6a-b55d495df54e None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:15.824359 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc99d2b-26b5-48ee-ac6a-b55d495df54e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:15.824708 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc99d2b-26b5-48ee-ac6a-b55d495df54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:15.879911 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-7fc99d2b-26b5-48ee-ac6a-b55d495df54e None None] MFA Rules not processed for user `22a9592143014f3287cd75c60ca17237`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:15.881232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-7fc99d2b-26b5-48ee-ac6a-b55d495df54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:15.976074 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 680/1359] 10.222.0.24 () {58 vars in 1058 bytes} [Fri May 27 03:27:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:27:15.984748 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382e515-dd6b-4104-8b6f-1319348f406c None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:15.999801 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382e515-dd6b-4104-8b6f-1319348f406c None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:16.005414 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebfb531-3a7e-48e2-ba24-72c721a27a67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:16.006153 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfb531-3a7e-48e2-ba24-72c721a27a67 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:16.006533 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfb531-3a7e-48e2-ba24-72c721a27a67 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:16.007055 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfb531-3a7e-48e2-ba24-72c721a27a67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:16.016560 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebfb531-3a7e-48e2-ba24-72c721a27a67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:16.018385 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebfb531-3a7e-48e2-ba24-72c721a27a67 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:16.029405 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 680/1360] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:27:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:27:18.502734 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105da63-c8f7-415a-b831-36f52148d958 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:18.503266 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105da63-c8f7-415a-b831-36f52148d958 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:18.503266 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105da63-c8f7-415a-b831-36f52148d958 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:18.504737 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 681/1361] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:18.509780 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316a1ce-80b7-4186-98c1-8ba911f4729b None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:18.510554 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316a1ce-80b7-4186-98c1-8ba911f4729b None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:18.511027 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316a1ce-80b7-4186-98c1-8ba911f4729b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:18.529766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-7316a1ce-80b7-4186-98c1-8ba911f4729b None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:18.531209 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-7316a1ce-80b7-4186-98c1-8ba911f4729b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:18.543542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 681/1362] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:27:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:18.582515 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105da63-c8f7-415a-b831-36f52148d958 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:18.597835 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105da63-c8f7-415a-b831-36f52148d958 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:18.603849 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67909304-a174-4bd5-af4b-111a23e7219d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:18.604465 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67909304-a174-4bd5-af4b-111a23e7219d service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:18.604745 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67909304-a174-4bd5-af4b-111a23e7219d service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:18.605018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67909304-a174-4bd5-af4b-111a23e7219d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:18.613786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67909304-a174-4bd5-af4b-111a23e7219d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:18.615405 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67909304-a174-4bd5-af4b-111a23e7219d service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:18.625436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 682/1363] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:27:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:27:21.288181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea79510b-b13e-46f9-ba46-d4abdc043885 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:21.289705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea79510b-b13e-46f9-ba46-d4abdc043885 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:21.290193 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea79510b-b13e-46f9-ba46-d4abdc043885 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:21.292119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 682/1364] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:27:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:21.299351 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4c293-f2d7-4ae1-992d-99dc4b3f3f51 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:21.299819 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4c293-f2d7-4ae1-992d-99dc4b3f3f51 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:21.300035 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4c293-f2d7-4ae1-992d-99dc4b3f3f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:21.322209 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-4ad4c293-f2d7-4ae1-992d-99dc4b3f3f51 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:21.323478 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4ad4c293-f2d7-4ae1-992d-99dc4b3f3f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:21.331640 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 683/1365] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:27:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:21.377781 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9f342-59c5-41b6-b9a9-794a28032276 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:21.378075 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9f342-59c5-41b6-b9a9-794a28032276 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:21.378358 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd9f342-59c5-41b6-b9a9-794a28032276 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:21.379771 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 683/1366] 10.222.0.24 () {54 vars in 961 bytes} [Fri May 27 03:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:21.385068 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eca023-6cbc-440e-8931-693fed449ea7 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:21.385381 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eca023-6cbc-440e-8931-693fed449ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:21.389370 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eca023-6cbc-440e-8931-693fed449ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:21.404098 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-34eca023-6cbc-440e-8931-693fed449ea7 None None] MFA Rules not processed for user `22a9592143014f3287cd75c60ca17237`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:21.405261 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-34eca023-6cbc-440e-8931-693fed449ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:21.415201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 684/1367] 10.222.0.24 () {58 vars in 1058 bytes} [Fri May 27 03:27:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:21.423059 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9f342-59c5-41b6-b9a9-794a28032276 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:21.437501 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd9f342-59c5-41b6-b9a9-794a28032276 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:21.442890 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b79a217-d883-4f27-9c89-014027dc3d49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:21.443520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79a217-d883-4f27-9c89-014027dc3d49 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:21.443800 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79a217-d883-4f27-9c89-014027dc3d49 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:21.444073 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79a217-d883-4f27-9c89-014027dc3d49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:21.452347 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b79a217-d883-4f27-9c89-014027dc3d49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:21.454200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b79a217-d883-4f27-9c89-014027dc3d49 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:21.464093 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 684/1368] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:27:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:27:23.875893 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25934d2d-2cd2-4b0e-9636-18bc5782b198 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:23.876805 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25934d2d-2cd2-4b0e-9636-18bc5782b198 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:23.877080 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25934d2d-2cd2-4b0e-9636-18bc5782b198 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:23.878610 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 685/1369] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:27:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:23.884213 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bc8d9-7d0e-411e-a10c-96d1c2499197 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:23.884696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bc8d9-7d0e-411e-a10c-96d1c2499197 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:23.884887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bc8d9-7d0e-411e-a10c-96d1c2499197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:23.904137 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-c94bc8d9-7d0e-411e-a10c-96d1c2499197 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:23.905332 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c94bc8d9-7d0e-411e-a10c-96d1c2499197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:23.913393 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 685/1370] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:23.958452 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25934d2d-2cd2-4b0e-9636-18bc5782b198 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:23.973516 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25934d2d-2cd2-4b0e-9636-18bc5782b198 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:23.984091 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8f626-826e-444b-9017-a9fe658d0894 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:23.985232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8f626-826e-444b-9017-a9fe658d0894 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:23.985736 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8f626-826e-444b-9017-a9fe658d0894 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:23.986111 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8f626-826e-444b-9017-a9fe658d0894 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:23.994439 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77523287-d196-46b1-be4e-d559ec44f17f None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:23.994707 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77523287-d196-46b1-be4e-d559ec44f17f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:23.995015 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77523287-d196-46b1-be4e-d559ec44f17f None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:23.996460 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 686/1371] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:27:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:23.999869 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8f626-826e-444b-9017-a9fe658d0894 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:24.003071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8f626-826e-444b-9017-a9fe658d0894 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:24.003529 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40eceef-7b48-4989-ab7b-a199744d44c3 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:24.003896 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40eceef-7b48-4989-ab7b-a199744d44c3 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:24.004383 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40eceef-7b48-4989-ab7b-a199744d44c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:24.015421 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 686/1372] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:27:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:27:24.027812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e40eceef-7b48-4989-ab7b-a199744d44c3 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:24.029047 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e40eceef-7b48-4989-ab7b-a199744d44c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:24.037813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 687/1373] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:27:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:27:24.356015 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6009768-0829-4be0-9750-b2698943996e None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:24.356015 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6009768-0829-4be0-9750-b2698943996e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:24.356015 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6009768-0829-4be0-9750-b2698943996e None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:24.358642 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 687/1374] 10.222.0.24 () {54 vars in 963 bytes} [Fri May 27 03:27:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:24.367512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c68a8-267c-4a27-b268-a5078a3fe579 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:24.367512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c68a8-267c-4a27-b268-a5078a3fe579 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:24.367512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c68a8-267c-4a27-b268-a5078a3fe579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:24.428255 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-774c68a8-267c-4a27-b268-a5078a3fe579 None None] MFA Rules not processed for user `c81ae2718faf4e59b799add42fb41c4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:24.428255 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-774c68a8-267c-4a27-b268-a5078a3fe579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:24.490929 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 688/1375] 10.222.0.24 () {58 vars in 1060 bytes} [Fri May 27 03:27:24 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:24.498788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6009768-0829-4be0-9750-b2698943996e None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:24.510156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6009768-0829-4be0-9750-b2698943996e None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:24.513655 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c085bc-879d-4aeb-8a2b-1149fbf3de1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:24.514296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c085bc-879d-4aeb-8a2b-1149fbf3de1e service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:24.514585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c085bc-879d-4aeb-8a2b-1149fbf3de1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:24.514883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c085bc-879d-4aeb-8a2b-1149fbf3de1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:24.524349 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c085bc-879d-4aeb-8a2b-1149fbf3de1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:24.525786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c085bc-879d-4aeb-8a2b-1149fbf3de1e service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:24.535506 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 688/1376] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:27:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:27:26.449518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574e8fc-2d6d-40ec-80ca-fe07daaefdf1 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:26.450085 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574e8fc-2d6d-40ec-80ca-fe07daaefdf1 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:26.450085 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574e8fc-2d6d-40ec-80ca-fe07daaefdf1 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:26.451582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 689/1377] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:26.457848 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee65be-9faa-4a60-95b7-96a6c137f2be None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:26.458374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee65be-9faa-4a60-95b7-96a6c137f2be None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:26.458501 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee65be-9faa-4a60-95b7-96a6c137f2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:26.477767 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-7aee65be-9faa-4a60-95b7-96a6c137f2be None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:26.479021 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-7aee65be-9faa-4a60-95b7-96a6c137f2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:26.487320 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 689/1378] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:27:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:26.532345 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574e8fc-2d6d-40ec-80ca-fe07daaefdf1 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:26.548752 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574e8fc-2d6d-40ec-80ca-fe07daaefdf1 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:26.554996 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9ae7f-9287-4f52-a0a6-bf78ce4d11a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:26.555642 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9ae7f-9287-4f52-a0a6-bf78ce4d11a9 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:26.555920 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9ae7f-9287-4f52-a0a6-bf78ce4d11a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:26.556185 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9ae7f-9287-4f52-a0a6-bf78ce4d11a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:26.564746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9ae7f-9287-4f52-a0a6-bf78ce4d11a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:26.566280 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9ae7f-9287-4f52-a0a6-bf78ce4d11a9 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:26.576960 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 690/1379] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:27:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:27:28.962304 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a48374-d0f3-497f-9137-f4b552c99fec None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:28.962867 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a48374-d0f3-497f-9137-f4b552c99fec None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:28.962867 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a48374-d0f3-497f-9137-f4b552c99fec None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:28.964239 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 690/1380] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:28.969556 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc8778-3cc2-4e6a-87b7-29b607f3a4ce None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:28.969996 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc8778-3cc2-4e6a-87b7-29b607f3a4ce None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:28.970106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc8778-3cc2-4e6a-87b7-29b607f3a4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:28.989260 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-4fcc8778-3cc2-4e6a-87b7-29b607f3a4ce None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:28.990690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-4fcc8778-3cc2-4e6a-87b7-29b607f3a4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:28.999381 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 691/1381] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:27:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:29.042442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a48374-d0f3-497f-9137-f4b552c99fec None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:29.057630 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a48374-d0f3-497f-9137-f4b552c99fec None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:29.063118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f973d-695f-4876-ab50-0e3445d0f479 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:29.063943 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f973d-695f-4876-ab50-0e3445d0f479 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:29.064332 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f973d-695f-4876-ab50-0e3445d0f479 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:29.064705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f973d-695f-4876-ab50-0e3445d0f479 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:29.073549 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f973d-695f-4876-ab50-0e3445d0f479 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:29.075502 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f973d-695f-4876-ab50-0e3445d0f479 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:29.080635 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:29.087791 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 691/1382] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:27:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:27:31.544046 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88650149-bd7b-46c6-a3c7-66cf724fb941 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:31.544572 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88650149-bd7b-46c6-a3c7-66cf724fb941 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:31.544572 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88650149-bd7b-46c6-a3c7-66cf724fb941 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:31.545949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 692/1383] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:31.551367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4217cd-2225-4b0e-85ae-a339d2d97389 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:31.551786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4217cd-2225-4b0e-85ae-a339d2d97389 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:31.551916 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4217cd-2225-4b0e-85ae-a339d2d97389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:31.570302 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-ec4217cd-2225-4b0e-85ae-a339d2d97389 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:31.571529 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ec4217cd-2225-4b0e-85ae-a339d2d97389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:31.579769 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 692/1384] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:31.621512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88650149-bd7b-46c6-a3c7-66cf724fb941 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:31.636010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88650149-bd7b-46c6-a3c7-66cf724fb941 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:31.641381 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f95234-b654-46b5-a98d-1666d7fbfb38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:31.642017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f95234-b654-46b5-a98d-1666d7fbfb38 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:31.642301 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f95234-b654-46b5-a98d-1666d7fbfb38 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:31.642572 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f95234-b654-46b5-a98d-1666d7fbfb38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:31.651032 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f95234-b654-46b5-a98d-1666d7fbfb38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:31.652617 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f95234-b654-46b5-a98d-1666d7fbfb38 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:31.662374 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 693/1385] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:27:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:27:33.906830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da84f50-bf51-432a-9fc6-314f34efc9e5 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:33.907409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da84f50-bf51-432a-9fc6-314f34efc9e5 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:33.907520 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da84f50-bf51-432a-9fc6-314f34efc9e5 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:33.908861 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 693/1386] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:33.914320 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116c4a0-7057-4527-b6da-a3be7210a913 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:33.914804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116c4a0-7057-4527-b6da-a3be7210a913 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:33.914991 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116c4a0-7057-4527-b6da-a3be7210a913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:33.936595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-c116c4a0-7057-4527-b6da-a3be7210a913 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:33.938075 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c116c4a0-7057-4527-b6da-a3be7210a913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:33.948892 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 694/1387] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:27:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:33.993707 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da84f50-bf51-432a-9fc6-314f34efc9e5 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:34.008869 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da84f50-bf51-432a-9fc6-314f34efc9e5 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:34.014411 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6292c-ecf2-42c3-bb6a-e8a1f2f9ce90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:34.015094 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6292c-ecf2-42c3-bb6a-e8a1f2f9ce90 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:34.015373 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6292c-ecf2-42c3-bb6a-e8a1f2f9ce90 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:34.015645 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6292c-ecf2-42c3-bb6a-e8a1f2f9ce90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:34.024540 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab6292c-ecf2-42c3-bb6a-e8a1f2f9ce90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:34.026119 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab6292c-ecf2-42c3-bb6a-e8a1f2f9ce90 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:34.035799 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 694/1388] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:27:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:27:36.564397 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099c286-fd99-4f76-8a99-cbbf92445e28 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:36.565091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099c286-fd99-4f76-8a99-cbbf92445e28 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:36.565091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099c286-fd99-4f76-8a99-cbbf92445e28 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:36.566386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 695/1389] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:36.572874 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d5a0c-7c02-4649-a634-79c663a1332c None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:36.573329 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d5a0c-7c02-4649-a634-79c663a1332c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:36.573578 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d5a0c-7c02-4649-a634-79c663a1332c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:36.595509 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-200d5a0c-7c02-4649-a634-79c663a1332c None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:36.597069 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-200d5a0c-7c02-4649-a634-79c663a1332c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:36.606795 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 695/1390] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:27:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:36.651958 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099c286-fd99-4f76-8a99-cbbf92445e28 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:36.666883 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4099c286-fd99-4f76-8a99-cbbf92445e28 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:36.672474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28bcd1b-ef6c-45a0-bc93-b9eeecd3e7de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:36.673136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bcd1b-ef6c-45a0-bc93-b9eeecd3e7de service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:36.673431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bcd1b-ef6c-45a0-bc93-b9eeecd3e7de service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:36.673720 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bcd1b-ef6c-45a0-bc93-b9eeecd3e7de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:36.682969 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28bcd1b-ef6c-45a0-bc93-b9eeecd3e7de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:36.684633 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28bcd1b-ef6c-45a0-bc93-b9eeecd3e7de service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:36.695199 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 696/1391] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:27:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:27:39.052816 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626c7fe-cacc-43f8-a034-eb79ef3413a5 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:39.053513 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626c7fe-cacc-43f8-a034-eb79ef3413a5 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:39.053513 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0626c7fe-cacc-43f8-a034-eb79ef3413a5 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:39.055576 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 696/1392] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:27:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:39.060505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293beda9-f035-477d-a800-d13713f017ad None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:39.060921 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293beda9-f035-477d-a800-d13713f017ad None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:39.061055 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293beda9-f035-477d-a800-d13713f017ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:39.078894 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-293beda9-f035-477d-a800-d13713f017ad None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:39.080084 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-293beda9-f035-477d-a800-d13713f017ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:39.088185 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 697/1393] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:27:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:39.132358 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626c7fe-cacc-43f8-a034-eb79ef3413a5 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:39.146486 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0626c7fe-cacc-43f8-a034-eb79ef3413a5 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:39.151880 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd0df1-d765-4890-9d0a-64bfe1c7e4e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:39.152511 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd0df1-d765-4890-9d0a-64bfe1c7e4e1 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:39.152794 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd0df1-d765-4890-9d0a-64bfe1c7e4e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:39.153071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd0df1-d765-4890-9d0a-64bfe1c7e4e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:39.161157 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd0df1-d765-4890-9d0a-64bfe1c7e4e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:39.162734 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd0df1-d765-4890-9d0a-64bfe1c7e4e1 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:39.172842 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 697/1394] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:27:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:27:40.592227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c8cb6-ad4c-4ce0-8c0c-b1b97986c1b0 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:40.592793 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c8cb6-ad4c-4ce0-8c0c-b1b97986c1b0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:40.592793 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801c8cb6-ad4c-4ce0-8c0c-b1b97986c1b0 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:40.594208 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 698/1395] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:40.599641 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a63d2-4663-46c0-a299-547b7f70e42a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:40.600072 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a63d2-4663-46c0-a299-547b7f70e42a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:40.600700 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18a63d2-4663-46c0-a299-547b7f70e42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:40.618729 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-d18a63d2-4663-46c0-a299-547b7f70e42a None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:40.619957 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d18a63d2-4663-46c0-a299-547b7f70e42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:40.627808 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 698/1396] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:27:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:40.683144 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb95c8-6d6d-433e-ba72-d06b29e71c4a None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:40.683509 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb95c8-6d6d-433e-ba72-d06b29e71c4a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:40.683861 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb95c8-6d6d-433e-ba72-d06b29e71c4a None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:40.685303 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 699/1397] 10.222.0.24 () {54 vars in 962 bytes} [Fri May 27 03:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:40.690157 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea50dc8-df18-4d19-aabf-045b912cd2bd None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:40.690446 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea50dc8-df18-4d19-aabf-045b912cd2bd None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:40.690725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea50dc8-df18-4d19-aabf-045b912cd2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:40.738745 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-4ea50dc8-df18-4d19-aabf-045b912cd2bd None None] MFA Rules not processed for user `95452d1b892a463d8e9e3eb16ca7c5ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:40.739968 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4ea50dc8-df18-4d19-aabf-045b912cd2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:40.800132 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 699/1398] 10.222.0.24 () {58 vars in 1059 bytes} [Fri May 27 03:27:40 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:40.808554 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb95c8-6d6d-433e-ba72-d06b29e71c4a None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:40.813840 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=87 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:40.819548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb95c8-6d6d-433e-ba72-d06b29e71c4a None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:40.823108 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a48cb-730f-4a71-96a0-f11f61cc456f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95452d1b892a463d8e9e3eb16ca7c5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:40.823808 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a48cb-730f-4a71-96a0-f11f61cc456f service glance] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:40.824176 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a48cb-730f-4a71-96a0-f11f61cc456f service glance] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:40.824439 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a48cb-730f-4a71-96a0-f11f61cc456f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:40.833539 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a48cb-730f-4a71-96a0-f11f61cc456f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:40.836681 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a48cb-730f-4a71-96a0-f11f61cc456f service glance] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:40.845483 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 700/1399] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:27:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:27:41.555565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e736a5f-7c33-44ab-bc3d-9c16d8761b20 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:41.555932 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e736a5f-7c33-44ab-bc3d-9c16d8761b20 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:41.556297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e736a5f-7c33-44ab-bc3d-9c16d8761b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:41.579932 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101567e-e1bb-4e49-bb50-66b94d732c69 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:41.580247 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101567e-e1bb-4e49-bb50-66b94d732c69 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:41.580524 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101567e-e1bb-4e49-bb50-66b94d732c69 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:41.581900 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 701/1400] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:27:41.587636 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24266dd4-2fc4-431a-b0fa-82c60a8a1dee None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:41.587923 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24266dd4-2fc4-431a-b0fa-82c60a8a1dee None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:41.588202 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24266dd4-2fc4-431a-b0fa-82c60a8a1dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:41.605185 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-24266dd4-2fc4-431a-b0fa-82c60a8a1dee None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:41.605774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-2e736a5f-7c33-44ab-bc3d-9c16d8761b20 None None] MFA Rules not processed for user `07c9688ca035484ba5341859574c2799`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:41.606381 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-24266dd4-2fc4-431a-b0fa-82c60a8a1dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:41.606743 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-2e736a5f-7c33-44ab-bc3d-9c16d8761b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:41.614245 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 702/1401] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:27:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:41.658822 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24266dd4-2fc4-431a-b0fa-82c60a8a1dee None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:41.668329 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:41.670844 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24266dd4-2fc4-431a-b0fa-82c60a8a1dee None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:41.672017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 700/1402] 10.222.0.24 () {58 vars in 1009 bytes} [Fri May 27 03:27:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:41.676344 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a39a2-8e11-4fae-ab31-6f4ca3226b11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:41.682081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a39a2-8e11-4fae-ab31-6f4ca3226b11 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:41.682081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a39a2-8e11-4fae-ab31-6f4ca3226b11 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:41.682081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a39a2-8e11-4fae-ab31-6f4ca3226b11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:41.686784 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a39a2-8e11-4fae-ab31-6f4ca3226b11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:41.688524 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a39a2-8e11-4fae-ab31-6f4ca3226b11 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:41.699184 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 703/1403] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:27:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:27:41.719135 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e651a-cd5a-417c-a04f-b8e6de7d672a None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:41.722580 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e651a-cd5a-417c-a04f-b8e6de7d672a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:41.722580 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9e651a-cd5a-417c-a04f-b8e6de7d672a None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:41.722580 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 701/1404] 10.222.0.24 () {54 vars in 942 bytes} [Fri May 27 03:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:41.727341 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f4f7a-d860-4009-b2ee-68158cce148f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:41.727634 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f4f7a-d860-4009-b2ee-68158cce148f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:41.727913 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f4f7a-d860-4009-b2ee-68158cce148f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:41.776584 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-c27f4f7a-d860-4009-b2ee-68158cce148f None None] MFA Rules not processed for user `21d559cd7127490da84185f20684929d`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:41.777837 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c27f4f7a-d860-4009-b2ee-68158cce148f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:41.849578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 704/1405] 10.222.0.24 () {58 vars in 1039 bytes} [Fri May 27 03:27:41 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:41.859120 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9e651a-cd5a-417c-a04f-b8e6de7d672a None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:41.873064 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9e651a-cd5a-417c-a04f-b8e6de7d672a None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:41.878026 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ea628-6338-4253-be14-6645a0e2c1f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d559cd7127490da84185f20684929d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:41.878654 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ea628-6338-4253-be14-6645a0e2c1f6 service swift] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:41.878974 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ea628-6338-4253-be14-6645a0e2c1f6 service swift] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:41.879248 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ea628-6338-4253-be14-6645a0e2c1f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:41.887802 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ea628-6338-4253-be14-6645a0e2c1f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:41.889442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ea628-6338-4253-be14-6645a0e2c1f6 service swift] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:41.897467 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 702/1406] 10.222.0.24 () {60 vars in 1384 bytes} [Fri May 27 03:27:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:27:44.138776 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88daa5-7791-4403-be80-63f57a8c31e9 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:44.139363 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88daa5-7791-4403-be80-63f57a8c31e9 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:44.139363 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88daa5-7791-4403-be80-63f57a8c31e9 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:44.140867 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 705/1407] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:27:44.148784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93cb65d-1e33-41c7-b745-ce2998c380a5 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:44.149338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93cb65d-1e33-41c7-b745-ce2998c380a5 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:44.149338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93cb65d-1e33-41c7-b745-ce2998c380a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:44.168760 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-c93cb65d-1e33-41c7-b745-ce2998c380a5 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:44.169973 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c93cb65d-1e33-41c7-b745-ce2998c380a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:44.178772 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 703/1408] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:44.239945 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88daa5-7791-4403-be80-63f57a8c31e9 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:44.257802 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88daa5-7791-4403-be80-63f57a8c31e9 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:44.263266 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e9ce84-18a2-4efc-b123-9cf3d0b02f1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:44.263924 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9ce84-18a2-4efc-b123-9cf3d0b02f1e service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:44.264210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9ce84-18a2-4efc-b123-9cf3d0b02f1e service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:44.264484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9ce84-18a2-4efc-b123-9cf3d0b02f1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:44.273712 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e9ce84-18a2-4efc-b123-9cf3d0b02f1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:44.275515 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e9ce84-18a2-4efc-b123-9cf3d0b02f1e service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:44.285213 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 706/1409] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:27:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:27:54.162522 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=9 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:54.280878 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=64 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:06.459855 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63453645-5a68-4ddf-9851-894b337305f8 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:06.460486 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63453645-5a68-4ddf-9851-894b337305f8 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:06.460486 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63453645-5a68-4ddf-9851-894b337305f8 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:06.461810 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 704/1410] 10.222.0.24 () {54 vars in 918 bytes} [Fri May 27 03:28:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:06.467185 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c485d1-c573-4bc1-8974-03e924027d1d None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:06.467626 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c485d1-c573-4bc1-8974-03e924027d1d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:06.467734 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c485d1-c573-4bc1-8974-03e924027d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:06.486668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-10c485d1-c573-4bc1-8974-03e924027d1d None None] MFA Rules not processed for user `95f2d0aceef4451a83ee89fac5130a0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:06.487880 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-10c485d1-c573-4bc1-8974-03e924027d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:06.496281 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 707/1411] 10.222.0.24 () {58 vars in 1015 bytes} [Fri May 27 03:28:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:06.510170 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80993db-f73a-40d9-b80f-ef5adc69bbe5 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:06.510462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80993db-f73a-40d9-b80f-ef5adc69bbe5 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:06.510743 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80993db-f73a-40d9-b80f-ef5adc69bbe5 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:06.512169 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 705/1412] 10.222.0.24 () {54 vars in 918 bytes} [Fri May 27 03:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:06.516539 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36482c-73f1-48f9-a600-d79e3d3f7a96 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:06.516827 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36482c-73f1-48f9-a600-d79e3d3f7a96 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:06.517100 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36482c-73f1-48f9-a600-d79e3d3f7a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:06.534706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-8e36482c-73f1-48f9-a600-d79e3d3f7a96 None None] MFA Rules not processed for user `22a9592143014f3287cd75c60ca17237`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:06.535922 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-8e36482c-73f1-48f9-a600-d79e3d3f7a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:06.545403 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 708/1413] 10.222.0.24 () {58 vars in 1015 bytes} [Fri May 27 03:28:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:09.189473 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46268bf3-cc90-4c76-9909-2ef1ed749e4d None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:09.190028 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46268bf3-cc90-4c76-9909-2ef1ed749e4d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:09.190028 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46268bf3-cc90-4c76-9909-2ef1ed749e4d None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:09.191468 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:09.196850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47754fb-cc06-4e1b-895a-71f534fc432a None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:09.197285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47754fb-cc06-4e1b-895a-71f534fc432a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:09.197384 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47754fb-cc06-4e1b-895a-71f534fc432a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:09.214760 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-f47754fb-cc06-4e1b-895a-71f534fc432a None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:09.216251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f47754fb-cc06-4e1b-895a-71f534fc432a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:09.223834 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:09.266337 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46268bf3-cc90-4c76-9909-2ef1ed749e4d None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:09.280739 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46268bf3-cc90-4c76-9909-2ef1ed749e4d None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:09.296597 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a66e30-50b4-4f8d-98b1-3d7a2b66c040 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:09.296597 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a66e30-50b4-4f8d-98b1-3d7a2b66c040 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:09.296597 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a66e30-50b4-4f8d-98b1-3d7a2b66c040 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:09.296597 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a66e30-50b4-4f8d-98b1-3d7a2b66c040 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:09.296974 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a66e30-50b4-4f8d-98b1-3d7a2b66c040 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:09.297105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a66e30-50b4-4f8d-98b1-3d7a2b66c040 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:09.306310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 707/1416] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:28:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:10.264586 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b254256-a7f3-450a-811b-0317fa811437 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:10.265119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b254256-a7f3-450a-811b-0317fa811437 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:10.265235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b254256-a7f3-450a-811b-0317fa811437 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:10.266596 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:10.271612 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c492ff-c656-43db-ae1a-fae69dfb9c97 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:10.272026 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c492ff-c656-43db-ae1a-fae69dfb9c97 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:10.272176 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c492ff-c656-43db-ae1a-fae69dfb9c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:10.289689 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-99c492ff-c656-43db-ae1a-fae69dfb9c97 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:10.290896 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-99c492ff-c656-43db-ae1a-fae69dfb9c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:10.298559 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:28:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:10.464028 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b254256-a7f3-450a-811b-0317fa811437 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:10.478367 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b254256-a7f3-450a-811b-0317fa811437 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:10.485149 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b02c8-8a9a-4cdd-8810-ceadfa48dea7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:10.486274 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b02c8-8a9a-4cdd-8810-ceadfa48dea7 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:10.486776 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b02c8-8a9a-4cdd-8810-ceadfa48dea7 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:10.487318 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b02c8-8a9a-4cdd-8810-ceadfa48dea7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:10.501396 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b02c8-8a9a-4cdd-8810-ceadfa48dea7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:10.503020 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b02c8-8a9a-4cdd-8810-ceadfa48dea7 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:10.512526 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 711/1419] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:28:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:28:11.463910 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82ac13-6a34-4b84-a622-2a4d564d8dd5 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:11.464355 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82ac13-6a34-4b84-a622-2a4d564d8dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:11.464737 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef82ac13-6a34-4b84-a622-2a4d564d8dd5 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:11.467832 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:28:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:11.471906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74978420-1c3c-4907-bcd3-469a9cef15d5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:11.472325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74978420-1c3c-4907-bcd3-469a9cef15d5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:11.472454 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74978420-1c3c-4907-bcd3-469a9cef15d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:11.491110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-74978420-1c3c-4907-bcd3-469a9cef15d5 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:11.492307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-74978420-1c3c-4907-bcd3-469a9cef15d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:11.500063 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:11.648148 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82ac13-6a34-4b84-a622-2a4d564d8dd5 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:11.661932 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef82ac13-6a34-4b84-a622-2a4d564d8dd5 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:11.667204 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5cb8af-15af-446f-9b4d-5b2293bf66e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:11.667827 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cb8af-15af-446f-9b4d-5b2293bf66e5 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:11.668106 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cb8af-15af-446f-9b4d-5b2293bf66e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:11.668378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5cb8af-15af-446f-9b4d-5b2293bf66e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:11.676246 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5cb8af-15af-446f-9b4d-5b2293bf66e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:11.678410 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5cb8af-15af-446f-9b4d-5b2293bf66e5 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:11.687622 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 710/1422] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:28:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:12.585770 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468a286-a886-4f3b-9c56-f70e2705fa58 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:12.586320 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468a286-a886-4f3b-9c56-f70e2705fa58 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:12.586320 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468a286-a886-4f3b-9c56-f70e2705fa58 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:12.587707 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:12.592563 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab623a-0f48-4f81-9635-0dd044be08a2 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:12.593013 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab623a-0f48-4f81-9635-0dd044be08a2 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:12.593122 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab623a-0f48-4f81-9635-0dd044be08a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:12.611266 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-60ab623a-0f48-4f81-9635-0dd044be08a2 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:12.612466 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-60ab623a-0f48-4f81-9635-0dd044be08a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:12.620028 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:28:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:12.766080 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468a286-a886-4f3b-9c56-f70e2705fa58 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:12.780829 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468a286-a886-4f3b-9c56-f70e2705fa58 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:12.786237 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16cf6b-dfed-40e4-a731-b9b0f32854d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:12.786918 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16cf6b-dfed-40e4-a731-b9b0f32854d4 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:12.787200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16cf6b-dfed-40e4-a731-b9b0f32854d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:12.787470 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16cf6b-dfed-40e4-a731-b9b0f32854d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:12.797148 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16cf6b-dfed-40e4-a731-b9b0f32854d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:12.798777 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16cf6b-dfed-40e4-a731-b9b0f32854d4 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:12.808241 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 714/1425] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:28:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:13.698888 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5c37f-108c-4460-bb9e-7ab0fb9343d5 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:13.699424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5c37f-108c-4460-bb9e-7ab0fb9343d5 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:13.699424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5c37f-108c-4460-bb9e-7ab0fb9343d5 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:13.700712 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:28:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:13.705783 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699dde98-150d-4ef2-8d15-729f09713f10 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:13.706205 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699dde98-150d-4ef2-8d15-729f09713f10 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:13.706361 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699dde98-150d-4ef2-8d15-729f09713f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:13.723691 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-699dde98-150d-4ef2-8d15-729f09713f10 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:13.724862 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-699dde98-150d-4ef2-8d15-729f09713f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:13.732561 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:28:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:13.880428 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de5c37f-108c-4460-bb9e-7ab0fb9343d5 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:13.894438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de5c37f-108c-4460-bb9e-7ab0fb9343d5 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:13.899699 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ed3f2-8d07-4d94-9425-774923a3dc5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:13.900320 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ed3f2-8d07-4d94-9425-774923a3dc5b service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:13.900598 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ed3f2-8d07-4d94-9425-774923a3dc5b service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:13.900867 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ed3f2-8d07-4d94-9425-774923a3dc5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:13.908806 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ed3f2-8d07-4d94-9425-774923a3dc5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:13.910511 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ed3f2-8d07-4d94-9425-774923a3dc5b service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:13.920416 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 713/1428] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:28:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:14.828377 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed4fe2-2d6f-415c-8d3d-ceabbf3c2c04 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:14.828937 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed4fe2-2d6f-415c-8d3d-ceabbf3c2c04 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:14.828937 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ed4fe2-2d6f-415c-8d3d-ceabbf3c2c04 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:14.830193 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:28:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:14.835151 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c137ebd-3117-45d4-9027-4d52eb2c0f89 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:14.835558 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c137ebd-3117-45d4-9027-4d52eb2c0f89 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:14.835692 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c137ebd-3117-45d4-9027-4d52eb2c0f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:14.854220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-5c137ebd-3117-45d4-9027-4d52eb2c0f89 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:14.855495 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5c137ebd-3117-45d4-9027-4d52eb2c0f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:14.863481 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:28:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:15.013062 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed4fe2-2d6f-415c-8d3d-ceabbf3c2c04 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:15.027527 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ed4fe2-2d6f-415c-8d3d-ceabbf3c2c04 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:15.032916 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47671db3-1309-4a03-b907-27464613eb86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:15.033547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47671db3-1309-4a03-b907-27464613eb86 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:15.033830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47671db3-1309-4a03-b907-27464613eb86 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:15.034107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47671db3-1309-4a03-b907-27464613eb86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:15.042169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47671db3-1309-4a03-b907-27464613eb86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:15.043788 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47671db3-1309-4a03-b907-27464613eb86 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:15.053082 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 717/1431] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:28:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:15.969268 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392ad9e-272d-460a-a2b5-22946c931871 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:15.969815 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392ad9e-272d-460a-a2b5-22946c931871 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:15.969941 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392ad9e-272d-460a-a2b5-22946c931871 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:15.971338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:28:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:15.976154 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70998aee-ff37-4948-9487-625b06a8cf5f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:15.976675 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70998aee-ff37-4948-9487-625b06a8cf5f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:15.976786 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70998aee-ff37-4948-9487-625b06a8cf5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:15.994649 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-70998aee-ff37-4948-9487-625b06a8cf5f None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:15.995856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-70998aee-ff37-4948-9487-625b06a8cf5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:16.003454 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:28:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:16.151675 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e392ad9e-272d-460a-a2b5-22946c931871 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:16.166780 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e392ad9e-272d-460a-a2b5-22946c931871 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:16.172188 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e2f572-c162-4112-81b1-00ada39849e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:16.172936 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e2f572-c162-4112-81b1-00ada39849e2 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:16.173223 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e2f572-c162-4112-81b1-00ada39849e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:16.173495 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e2f572-c162-4112-81b1-00ada39849e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:16.181953 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e2f572-c162-4112-81b1-00ada39849e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:16.183556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e2f572-c162-4112-81b1-00ada39849e2 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:16.192922 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 716/1434] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:28:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:17.174176 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcf7fb-ec82-479e-a9db-4169106c78b6 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:17.174723 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcf7fb-ec82-479e-a9db-4169106c78b6 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:17.174723 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bcf7fb-ec82-479e-a9db-4169106c78b6 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:17.176126 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:28:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:17.181506 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7bfab-b9eb-4b77-9ef1-fba9b17d6493 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:17.181930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7bfab-b9eb-4b77-9ef1-fba9b17d6493 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:17.182043 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7bfab-b9eb-4b77-9ef1-fba9b17d6493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:17.200225 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-cbb7bfab-b9eb-4b77-9ef1-fba9b17d6493 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:17.201341 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-cbb7bfab-b9eb-4b77-9ef1-fba9b17d6493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:17.209286 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:17.357504 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bcf7fb-ec82-479e-a9db-4169106c78b6 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:17.372015 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bcf7fb-ec82-479e-a9db-4169106c78b6 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:17.377416 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70210922-1346-42cc-b073-9714ff564553 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:17.378037 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70210922-1346-42cc-b073-9714ff564553 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:17.378315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70210922-1346-42cc-b073-9714ff564553 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:17.378585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70210922-1346-42cc-b073-9714ff564553 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:17.387077 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70210922-1346-42cc-b073-9714ff564553 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:17.388690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70210922-1346-42cc-b073-9714ff564553 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:17.398368 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 720/1437] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:28:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:18.314056 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01f949-79b4-4d44-a0c4-be2f32b5f371 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:18.314638 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01f949-79b4-4d44-a0c4-be2f32b5f371 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:18.314638 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01f949-79b4-4d44-a0c4-be2f32b5f371 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:18.316094 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:18.321542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0a927-c8ab-47c2-9437-552152d160ff None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:18.322025 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0a927-c8ab-47c2-9437-552152d160ff None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:18.322131 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0a927-c8ab-47c2-9437-552152d160ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:18.340612 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-81d0a927-c8ab-47c2-9437-552152d160ff None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:18.341739 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-81d0a927-c8ab-47c2-9437-552152d160ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:18.349405 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:28:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:18.498089 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01f949-79b4-4d44-a0c4-be2f32b5f371 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:18.512849 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01f949-79b4-4d44-a0c4-be2f32b5f371 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:18.518223 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4f2c8-f2f5-491a-88ae-8eaad5a8b282 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:18.518860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4f2c8-f2f5-491a-88ae-8eaad5a8b282 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:18.519155 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4f2c8-f2f5-491a-88ae-8eaad5a8b282 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:18.519425 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4f2c8-f2f5-491a-88ae-8eaad5a8b282 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:18.528040 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d4f2c8-f2f5-491a-88ae-8eaad5a8b282 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:18.529609 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d4f2c8-f2f5-491a-88ae-8eaad5a8b282 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:18.538887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 719/1440] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:28:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:20.621757 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a059898-3865-441a-ac21-2d4701419db8 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:20.622359 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a059898-3865-441a-ac21-2d4701419db8 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:20.622359 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a059898-3865-441a-ac21-2d4701419db8 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:20.623709 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:20.629017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1718333-f28c-4760-86f9-3fe93f9c205a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:20.629426 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1718333-f28c-4760-86f9-3fe93f9c205a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:20.629573 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1718333-f28c-4760-86f9-3fe93f9c205a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:20.646888 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-e1718333-f28c-4760-86f9-3fe93f9c205a None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:20.648104 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e1718333-f28c-4760-86f9-3fe93f9c205a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:20.655780 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:20.693598 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a059898-3865-441a-ac21-2d4701419db8 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:20.704272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a059898-3865-441a-ac21-2d4701419db8 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:20.707708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b2ca6f-cb26-4ce3-9de2-b4c308ad1255 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:20.708327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2ca6f-cb26-4ce3-9de2-b4c308ad1255 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:20.708601 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2ca6f-cb26-4ce3-9de2-b4c308ad1255 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:20.708870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b2ca6f-cb26-4ce3-9de2-b4c308ad1255 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:20.717455 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b2ca6f-cb26-4ce3-9de2-b4c308ad1255 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:20.718901 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b2ca6f-cb26-4ce3-9de2-b4c308ad1255 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:20.728674 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 723/1443] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:28:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:23.168484 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac32da-8006-4eee-a487-247c0d736ae8 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:23.169045 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac32da-8006-4eee-a487-247c0d736ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:23.169045 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac32da-8006-4eee-a487-247c0d736ae8 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:23.170393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:23.175762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b7caa6-cec6-485f-8236-51ae8fa5dd92 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:23.176147 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b7caa6-cec6-485f-8236-51ae8fa5dd92 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:23.176315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b7caa6-cec6-485f-8236-51ae8fa5dd92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:23.193786 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-76b7caa6-cec6-485f-8236-51ae8fa5dd92 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:23.195024 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-76b7caa6-cec6-485f-8236-51ae8fa5dd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:23.202650 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:28:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:23.241001 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac32da-8006-4eee-a487-247c0d736ae8 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:23.242686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b6ac32da-8006-4eee-a487-247c0d736ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:23.255348 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac32da-8006-4eee-a487-247c0d736ae8 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:23.258789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ed2a4-f141-4f0c-8f8c-0ead1487b865 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:23.259438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ed2a4-f141-4f0c-8f8c-0ead1487b865 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:23.259720 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ed2a4-f141-4f0c-8f8c-0ead1487b865 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:23.259995 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ed2a4-f141-4f0c-8f8c-0ead1487b865 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:23.268670 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ed2a4-f141-4f0c-8f8c-0ead1487b865 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:23.270081 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ed2a4-f141-4f0c-8f8c-0ead1487b865 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:23.279914 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 722/1446] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:28:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:25.868048 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f42ef1-6a1f-4fc8-8d09-cf1c13be0762 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:25.868692 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f42ef1-6a1f-4fc8-8d09-cf1c13be0762 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:25.868692 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f42ef1-6a1f-4fc8-8d09-cf1c13be0762 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:25.869940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:28:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:25.875312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557121c1-5ec9-4b5a-b8e3-0df1ab6c104e None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:25.875773 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557121c1-5ec9-4b5a-b8e3-0df1ab6c104e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:25.875890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557121c1-5ec9-4b5a-b8e3-0df1ab6c104e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:25.893125 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-557121c1-5ec9-4b5a-b8e3-0df1ab6c104e None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:25.894316 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-557121c1-5ec9-4b5a-b8e3-0df1ab6c104e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:25.902153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:25.940948 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f42ef1-6a1f-4fc8-8d09-cf1c13be0762 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:25.951330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f42ef1-6a1f-4fc8-8d09-cf1c13be0762 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:25.955100 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26cefe-bb3f-416c-bc01-afff0bafd10a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:25.955721 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26cefe-bb3f-416c-bc01-afff0bafd10a service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:25.955998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26cefe-bb3f-416c-bc01-afff0bafd10a service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:25.956266 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26cefe-bb3f-416c-bc01-afff0bafd10a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:25.964842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26cefe-bb3f-416c-bc01-afff0bafd10a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:25.966255 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26cefe-bb3f-416c-bc01-afff0bafd10a service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:25.975602 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 726/1449] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:28:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:28.313051 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aea3b4-5e4c-430d-a1aa-d4bc81ede576 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:28.313609 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aea3b4-5e4c-430d-a1aa-d4bc81ede576 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:28.313609 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aea3b4-5e4c-430d-a1aa-d4bc81ede576 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:28.315001 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:28.320320 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec73e2-7ffc-4f98-81a2-91b6f59ed158 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:28.320730 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec73e2-7ffc-4f98-81a2-91b6f59ed158 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:28.320911 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec73e2-7ffc-4f98-81a2-91b6f59ed158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:28.338927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-1cec73e2-7ffc-4f98-81a2-91b6f59ed158 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:28.340104 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1cec73e2-7ffc-4f98-81a2-91b6f59ed158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:28.348009 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:28:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:28.386042 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aea3b4-5e4c-430d-a1aa-d4bc81ede576 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:28.396480 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aea3b4-5e4c-430d-a1aa-d4bc81ede576 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:28.400432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074d201-71bc-4283-a308-9d783b87ca9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:28.401055 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074d201-71bc-4283-a308-9d783b87ca9c service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:28.401334 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074d201-71bc-4283-a308-9d783b87ca9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:28.401608 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074d201-71bc-4283-a308-9d783b87ca9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:28.410350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5074d201-71bc-4283-a308-9d783b87ca9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:28.411802 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5074d201-71bc-4283-a308-9d783b87ca9c service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:28.421482 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 725/1452] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:28:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:30.639700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0f788-398e-48d9-ba8a-f4cf324101ea None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:30.640299 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0f788-398e-48d9-ba8a-f4cf324101ea None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:30.640299 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0f788-398e-48d9-ba8a-f4cf324101ea None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:30.641904 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:28:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:30.648222 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c6b79-8bcd-4d21-a83d-4b67c2dec035 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:30.648679 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c6b79-8bcd-4d21-a83d-4b67c2dec035 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:30.648778 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c6b79-8bcd-4d21-a83d-4b67c2dec035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:30.669245 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-fa0c6b79-8bcd-4d21-a83d-4b67c2dec035 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:30.671039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-fa0c6b79-8bcd-4d21-a83d-4b67c2dec035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:30.680355 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:28:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:30.720977 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0f788-398e-48d9-ba8a-f4cf324101ea None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:30.732472 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0f788-398e-48d9-ba8a-f4cf324101ea None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:30.736044 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ec2ee4-96ef-4c21-9518-a0cccde35674 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:30.736687 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec2ee4-96ef-4c21-9518-a0cccde35674 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:30.737000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec2ee4-96ef-4c21-9518-a0cccde35674 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:30.737280 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ec2ee4-96ef-4c21-9518-a0cccde35674 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:30.747594 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ec2ee4-96ef-4c21-9518-a0cccde35674 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:30.749537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ec2ee4-96ef-4c21-9518-a0cccde35674 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:30.759704 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 729/1455] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:28:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:33.016716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc8162-a2c2-4465-903c-4be821cf4e37 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:33.017296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc8162-a2c2-4465-903c-4be821cf4e37 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:33.017296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc8162-a2c2-4465-903c-4be821cf4e37 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:33.018621 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:33.023762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee8357-d348-43a3-9c01-3cd1fb26c527 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:33.024201 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee8357-d348-43a3-9c01-3cd1fb26c527 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:33.024319 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee8357-d348-43a3-9c01-3cd1fb26c527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:33.041801 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-b6ee8357-d348-43a3-9c01-3cd1fb26c527 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:33.043012 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b6ee8357-d348-43a3-9c01-3cd1fb26c527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:33.050633 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:28:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:33.088216 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc8162-a2c2-4465-903c-4be821cf4e37 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:33.098809 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc8162-a2c2-4465-903c-4be821cf4e37 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:33.102309 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f70970-25bb-4370-b424-17f814b4ad47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:33.102970 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f70970-25bb-4370-b424-17f814b4ad47 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:33.103255 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f70970-25bb-4370-b424-17f814b4ad47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:33.103529 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f70970-25bb-4370-b424-17f814b4ad47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:33.112175 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f70970-25bb-4370-b424-17f814b4ad47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:33.113611 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f70970-25bb-4370-b424-17f814b4ad47 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:33.123241 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 728/1458] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:28:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:35.333255 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb19b2b-5bd7-4006-b329-562150863d02 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:35.333255 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb19b2b-5bd7-4006-b329-562150863d02 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:35.333255 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb19b2b-5bd7-4006-b329-562150863d02 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:35.333255 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:35.338641 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45927764-da4f-43a8-b783-25d3b7977dff None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:35.339124 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45927764-da4f-43a8-b783-25d3b7977dff None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:35.339230 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45927764-da4f-43a8-b783-25d3b7977dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:35.345364 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=70 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:35.357369 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-45927764-da4f-43a8-b783-25d3b7977dff None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:35.358554 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-45927764-da4f-43a8-b783-25d3b7977dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:35.367048 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:35.403596 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb19b2b-5bd7-4006-b329-562150863d02 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:35.408927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=73 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:35.414582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb19b2b-5bd7-4006-b329-562150863d02 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:35.418148 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca5bd9-5200-4b68-9de8-2f4db2b0c445 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:35.422476 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca5bd9-5200-4b68-9de8-2f4db2b0c445 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:35.422476 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca5bd9-5200-4b68-9de8-2f4db2b0c445 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:35.422476 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca5bd9-5200-4b68-9de8-2f4db2b0c445 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:35.428094 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beca5bd9-5200-4b68-9de8-2f4db2b0c445 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:35.430017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beca5bd9-5200-4b68-9de8-2f4db2b0c445 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:35.439166 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 732/1461] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:28:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:37.673779 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afa9e3-ba73-457b-b113-a3049904a3d9 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:37.674339 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afa9e3-ba73-457b-b113-a3049904a3d9 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:37.674339 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afa9e3-ba73-457b-b113-a3049904a3d9 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:37.675741 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:28:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:37.680935 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79e982-7078-4c1a-91d7-21100810f026 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:37.681375 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79e982-7078-4c1a-91d7-21100810f026 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:37.681491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79e982-7078-4c1a-91d7-21100810f026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:37.698692 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-5b79e982-7078-4c1a-91d7-21100810f026 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:37.699908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-5b79e982-7078-4c1a-91d7-21100810f026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:37.707909 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:37.745031 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2afa9e3-ba73-457b-b113-a3049904a3d9 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:37.755641 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2afa9e3-ba73-457b-b113-a3049904a3d9 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:37.759634 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df3820c-7661-4005-9438-55dc8edbe7de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:37.760257 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df3820c-7661-4005-9438-55dc8edbe7de service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:37.760539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df3820c-7661-4005-9438-55dc8edbe7de service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:37.760813 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df3820c-7661-4005-9438-55dc8edbe7de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:37.769507 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df3820c-7661-4005-9438-55dc8edbe7de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:37.770973 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df3820c-7661-4005-9438-55dc8edbe7de service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:37.780239 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 731/1464] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:28:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:39.964552 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79cdf35-6d6b-4d91-934d-629bc639e2fd None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:39.965092 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79cdf35-6d6b-4d91-934d-629bc639e2fd None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:39.965092 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79cdf35-6d6b-4d91-934d-629bc639e2fd None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:39.966487 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:39.971687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e091b-0a18-491a-a394-a5f451e14987 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:39.972126 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e091b-0a18-491a-a394-a5f451e14987 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:39.972271 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e091b-0a18-491a-a394-a5f451e14987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:39.989836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-096e091b-0a18-491a-a394-a5f451e14987 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:39.991601 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-096e091b-0a18-491a-a394-a5f451e14987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:39.999632 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:40.035795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79cdf35-6d6b-4d91-934d-629bc639e2fd None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:40.046927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79cdf35-6d6b-4d91-934d-629bc639e2fd None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:40.050422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d6451-a1a0-402a-a9a7-66276d235c59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:40.051084 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d6451-a1a0-402a-a9a7-66276d235c59 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:40.051370 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d6451-a1a0-402a-a9a7-66276d235c59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:40.051647 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d6451-a1a0-402a-a9a7-66276d235c59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:40.061076 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d6451-a1a0-402a-a9a7-66276d235c59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:40.062519 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d6451-a1a0-402a-a9a7-66276d235c59 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:40.071805 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 735/1467] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:28:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:42.308306 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0424cc-c569-4c6b-a363-d62558e5ccda None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:42.308831 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0424cc-c569-4c6b-a363-d62558e5ccda None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:42.308831 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0424cc-c569-4c6b-a363-d62558e5ccda None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:42.310241 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:28:42.315569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e226bd0-c369-4bb2-abb2-e9db40e8a59f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:42.316026 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e226bd0-c369-4bb2-abb2-e9db40e8a59f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:42.316133 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e226bd0-c369-4bb2-abb2-e9db40e8a59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:42.334325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-8e226bd0-c369-4bb2-abb2-e9db40e8a59f None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:42.335996 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-8e226bd0-c369-4bb2-abb2-e9db40e8a59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:42.343756 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:42.379788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0424cc-c569-4c6b-a363-d62558e5ccda None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:42.391191 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0424cc-c569-4c6b-a363-d62558e5ccda None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:42.394651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cf7fc-bf7d-4e0a-9486-2d8c0a293fc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:42.395322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cf7fc-bf7d-4e0a-9486-2d8c0a293fc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:42.395609 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cf7fc-bf7d-4e0a-9486-2d8c0a293fc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:42.395885 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cf7fc-bf7d-4e0a-9486-2d8c0a293fc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:42.405368 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47cf7fc-bf7d-4e0a-9486-2d8c0a293fc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:42.406795 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47cf7fc-bf7d-4e0a-9486-2d8c0a293fc8 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:42.416110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 734/1470] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:28:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:52.335890 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=17 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:52.411938 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=17 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:15.334879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c6af1-0fc6-4bf0-8a45-9554fd1cff3b None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:15.334879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c6af1-0fc6-4bf0-8a45-9554fd1cff3b None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:15.335544 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c6af1-0fc6-4bf0-8a45-9554fd1cff3b None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:15.336442 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 737/1471] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:15.341721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05e7f6-a27a-49e1-8415-74066b59d8df None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:15.342147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05e7f6-a27a-49e1-8415-74066b59d8df None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:15.342272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b05e7f6-a27a-49e1-8415-74066b59d8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:15.360426 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-8b05e7f6-a27a-49e1-8415-74066b59d8df None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:15.363587 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8b05e7f6-a27a-49e1-8415-74066b59d8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:15.370238 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 735/1472] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:15.416505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cfe6fb-4b74-4b4b-9fd2-c26291b68f8f None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:15.416786 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cfe6fb-4b74-4b4b-9fd2-c26291b68f8f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:15.417056 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cfe6fb-4b74-4b4b-9fd2-c26291b68f8f None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:15.418399 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 738/1473] 10.222.0.24 () {54 vars in 962 bytes} [Fri May 27 03:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:15.423157 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd1fcf-f3aa-4ded-8fb2-cb5354f980c2 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:15.423449 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd1fcf-f3aa-4ded-8fb2-cb5354f980c2 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:15.423724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd1fcf-f3aa-4ded-8fb2-cb5354f980c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:15.442385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-0fdd1fcf-f3aa-4ded-8fb2-cb5354f980c2 None None] MFA Rules not processed for user `95452d1b892a463d8e9e3eb16ca7c5ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:15.443599 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-0fdd1fcf-f3aa-4ded-8fb2-cb5354f980c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:15.449838 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 736/1474] 10.222.0.24 () {58 vars in 1059 bytes} [Fri May 27 03:29:15 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:15.457026 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cfe6fb-4b74-4b4b-9fd2-c26291b68f8f None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:15.467319 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cfe6fb-4b74-4b4b-9fd2-c26291b68f8f None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:15.470738 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27598b87-a37f-486d-ac31-f286b0677542 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95452d1b892a463d8e9e3eb16ca7c5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:15.471380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27598b87-a37f-486d-ac31-f286b0677542 service glance] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:15.471662 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27598b87-a37f-486d-ac31-f286b0677542 service glance] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:15.471933 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27598b87-a37f-486d-ac31-f286b0677542 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:15.481302 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27598b87-a37f-486d-ac31-f286b0677542 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:15.482723 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27598b87-a37f-486d-ac31-f286b0677542 service glance] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:15.492324 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 739/1475] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:29:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:17.777788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc31dca-1921-499d-b469-7452063ad9d6 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:17.778380 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc31dca-1921-499d-b469-7452063ad9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:17.778380 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc31dca-1921-499d-b469-7452063ad9d6 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:17.779733 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 737/1476] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:17.785055 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d9b38-d172-49f3-99ef-903a32464630 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:17.786443 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d9b38-d172-49f3-99ef-903a32464630 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:17.786443 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d9b38-d172-49f3-99ef-903a32464630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:17.803798 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-6e2d9b38-d172-49f3-99ef-903a32464630 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:17.804992 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6e2d9b38-d172-49f3-99ef-903a32464630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:17.812758 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 740/1477] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:29:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:17.837603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc31dca-1921-499d-b469-7452063ad9d6 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:17.849908 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc31dca-1921-499d-b469-7452063ad9d6 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:17.854494 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca46c3c-b13a-40db-be7f-eb68a2ccde09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:17.855180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca46c3c-b13a-40db-be7f-eb68a2ccde09 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:17.855456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca46c3c-b13a-40db-be7f-eb68a2ccde09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:17.855726 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca46c3c-b13a-40db-be7f-eb68a2ccde09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:17.856509 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca46c3c-b13a-40db-be7f-eb68a2ccde09 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:17.858324 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: WARNING py.warnings [None req-3ca46c3c-b13a-40db-be7f-eb68a2ccde09 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:29:17.858324 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: warnings.warn(msg) May 27 03:29:17.858324 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: May 27 03:29:17.859508 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca46c3c-b13a-40db-be7f-eb68a2ccde09 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:17.878760 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 738/1478] 10.222.0.24 () {56 vars in 1108 bytes} [Fri May 27 03:29:17 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:29:20.075249 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f24009-b2e3-4835-a250-05af0daccebc None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:20.075811 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f24009-b2e3-4835-a250-05af0daccebc None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:20.075811 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f24009-b2e3-4835-a250-05af0daccebc None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:20.077159 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 741/1479] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:20.083504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c16aa-2e60-4db7-8d1e-7cf5a2694306 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:20.083952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c16aa-2e60-4db7-8d1e-7cf5a2694306 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:20.084108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c16aa-2e60-4db7-8d1e-7cf5a2694306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:20.101324 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-d98c16aa-2e60-4db7-8d1e-7cf5a2694306 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:20.102500 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d98c16aa-2e60-4db7-8d1e-7cf5a2694306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:20.110073 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 739/1480] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:29:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:20.152916 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f24009-b2e3-4835-a250-05af0daccebc None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:20.167104 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f24009-b2e3-4835-a250-05af0daccebc None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:20.172422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d593e-9f01-4213-b916-5c2559fc6fb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:20.173050 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d593e-9f01-4213-b916-5c2559fc6fb4 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:20.173331 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d593e-9f01-4213-b916-5c2559fc6fb4 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:20.173604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d593e-9f01-4213-b916-5c2559fc6fb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:20.182061 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d593e-9f01-4213-b916-5c2559fc6fb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:20.183681 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d593e-9f01-4213-b916-5c2559fc6fb4 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:20.193555 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 742/1481] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:29:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:22.545984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e380b-5c85-41ec-84b8-8fe7c0c2bcc9 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:22.546538 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e380b-5c85-41ec-84b8-8fe7c0c2bcc9 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:22.546538 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e380b-5c85-41ec-84b8-8fe7c0c2bcc9 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:22.547948 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 740/1482] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:22.553824 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52d20b-5b68-441a-a820-68e6a6b1e8ff None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:22.554248 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52d20b-5b68-441a-a820-68e6a6b1e8ff None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:22.554380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52d20b-5b68-441a-a820-68e6a6b1e8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:22.573379 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-8b52d20b-5b68-441a-a820-68e6a6b1e8ff None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:22.574605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-8b52d20b-5b68-441a-a820-68e6a6b1e8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:22.585925 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 743/1483] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:29:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:22.624588 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e380b-5c85-41ec-84b8-8fe7c0c2bcc9 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:22.636028 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e380b-5c85-41ec-84b8-8fe7c0c2bcc9 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:22.639640 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c948922-a6c0-456d-89fc-cb19a6a4a018 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:22.640277 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c948922-a6c0-456d-89fc-cb19a6a4a018 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:22.640561 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c948922-a6c0-456d-89fc-cb19a6a4a018 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:22.640837 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c948922-a6c0-456d-89fc-cb19a6a4a018 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:22.649789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c948922-a6c0-456d-89fc-cb19a6a4a018 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:22.651261 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c948922-a6c0-456d-89fc-cb19a6a4a018 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:22.660904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 741/1484] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:29:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:24.911530 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a0949-7b04-4875-8e21-22c68beb38d7 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:24.912186 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a0949-7b04-4875-8e21-22c68beb38d7 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:24.912186 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a0949-7b04-4875-8e21-22c68beb38d7 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:24.913418 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 744/1485] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:24.918586 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd159a2-6459-4b88-91ad-b9570d2b5280 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:24.919072 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd159a2-6459-4b88-91ad-b9570d2b5280 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:24.919307 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd159a2-6459-4b88-91ad-b9570d2b5280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:24.937570 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-ddd159a2-6459-4b88-91ad-b9570d2b5280 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:24.938765 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ddd159a2-6459-4b88-91ad-b9570d2b5280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:24.947871 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 742/1486] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:29:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:24.984963 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a0949-7b04-4875-8e21-22c68beb38d7 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:24.995937 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a0949-7b04-4875-8e21-22c68beb38d7 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:24.999381 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2aacf8-8f07-455b-994d-685703786cd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:25.000007 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2aacf8-8f07-455b-994d-685703786cd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:25.000286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2aacf8-8f07-455b-994d-685703786cd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:25.000558 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2aacf8-8f07-455b-994d-685703786cd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:25.009977 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2aacf8-8f07-455b-994d-685703786cd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:25.011450 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2aacf8-8f07-455b-994d-685703786cd1 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:25.020919 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 745/1487] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:29:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:27.297334 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097dc57-68b0-4483-88e0-012ab743d693 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:27.297975 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097dc57-68b0-4483-88e0-012ab743d693 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:27.297975 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097dc57-68b0-4483-88e0-012ab743d693 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:27.299304 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 743/1488] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:29:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:27.304587 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcfc6e-65d1-4473-9282-fc5913e642db None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:27.305001 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcfc6e-65d1-4473-9282-fc5913e642db None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:27.305160 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcfc6e-65d1-4473-9282-fc5913e642db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:27.323096 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-6ffcfc6e-65d1-4473-9282-fc5913e642db None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:27.324283 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6ffcfc6e-65d1-4473-9282-fc5913e642db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:27.334924 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 746/1489] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:27.371566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7097dc57-68b0-4483-88e0-012ab743d693 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:27.383095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7097dc57-68b0-4483-88e0-012ab743d693 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:27.386669 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2efee6-ff13-4c14-916f-eee74b1b5473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:27.387231 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2efee6-ff13-4c14-916f-eee74b1b5473 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:27.387546 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2efee6-ff13-4c14-916f-eee74b1b5473 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:27.387935 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2efee6-ff13-4c14-916f-eee74b1b5473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:27.397714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2efee6-ff13-4c14-916f-eee74b1b5473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:27.399216 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2efee6-ff13-4c14-916f-eee74b1b5473 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:27.408636 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 744/1490] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:29:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:30.023262 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2da63-cece-449f-a5ad-bebc80990722 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:30.023829 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2da63-cece-449f-a5ad-bebc80990722 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:30.023829 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2da63-cece-449f-a5ad-bebc80990722 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:30.025177 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 747/1491] 10.222.0.24 () {54 vars in 926 bytes} [Fri May 27 03:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:30.030265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dcd46-dcec-4e78-ab4e-d459c98a2a87 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:30.030693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dcd46-dcec-4e78-ab4e-d459c98a2a87 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:30.030812 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dcd46-dcec-4e78-ab4e-d459c98a2a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:30.048964 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-6b1dcd46-dcec-4e78-ab4e-d459c98a2a87 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:30.050155 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6b1dcd46-dcec-4e78-ab4e-d459c98a2a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:30.057979 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 745/1492] 10.222.0.24 () {58 vars in 1023 bytes} [Fri May 27 03:29:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:30.096671 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2da63-cece-449f-a5ad-bebc80990722 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:30.107321 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2da63-cece-449f-a5ad-bebc80990722 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:30.111040 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6331d5d3-0ce6-4395-87f4-0c98f7db44fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:30.111677 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331d5d3-0ce6-4395-87f4-0c98f7db44fd service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:30.111956 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331d5d3-0ce6-4395-87f4-0c98f7db44fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:30.112228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6331d5d3-0ce6-4395-87f4-0c98f7db44fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:30.121638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331d5d3-0ce6-4395-87f4-0c98f7db44fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:30.123103 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6331d5d3-0ce6-4395-87f4-0c98f7db44fd service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:30.132412 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 748/1493] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:29:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:40.043978 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=33 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:40.128217 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=33 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:20.827003 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ba390-a6fd-49c2-91cf-30eab3f5f09b None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:20.828194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ba390-a6fd-49c2-91cf-30eab3f5f09b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:20.828194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ba390-a6fd-49c2-91cf-30eab3f5f09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:20.850903 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-537ba390-a6fd-49c2-91cf-30eab3f5f09b None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:20.852141 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-537ba390-a6fd-49c2-91cf-30eab3f5f09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:20.860562 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 746/1494] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:31:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:20.869959 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f72f5-bc53-4d51-8427-7c342eca0c7c None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:20.870504 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f72f5-bc53-4d51-8427-7c342eca0c7c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:20.870504 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f72f5-bc53-4d51-8427-7c342eca0c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:20.890085 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-783f72f5-bc53-4d51-8427-7c342eca0c7c None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:20.891322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-783f72f5-bc53-4d51-8427-7c342eca0c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:20.900343 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 749/1495] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:31:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:20.904986 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ba390-a6fd-49c2-91cf-30eab3f5f09b None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:20.914701 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ba390-a6fd-49c2-91cf-30eab3f5f09b None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:20.919268 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ac70c-2b9d-493c-a59b-3d441cef9229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:20.919917 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ac70c-2b9d-493c-a59b-3d441cef9229 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:20.920201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ac70c-2b9d-493c-a59b-3d441cef9229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:20.920479 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ac70c-2b9d-493c-a59b-3d441cef9229 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:20.921205 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ac70c-2b9d-493c-a59b-3d441cef9229 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:31:20.921514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ac70c-2b9d-493c-a59b-3d441cef9229 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:20.923109 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: WARNING py.warnings [None req-435ac70c-2b9d-493c-a59b-3d441cef9229 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 03:31:20.923109 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: warnings.warn(msg) May 27 03:31:20.923109 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: May 27 03:31:20.923992 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ac70c-2b9d-493c-a59b-3d441cef9229 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:20.935700 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 747/1496] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:31:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:20.944396 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f72f5-bc53-4d51-8427-7c342eca0c7c None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:20.953730 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f72f5-bc53-4d51-8427-7c342eca0c7c None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:20.958236 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cd695-5daf-4c2a-b2af-3c734677aa2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:20.958912 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cd695-5daf-4c2a-b2af-3c734677aa2c admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:20.959198 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cd695-5daf-4c2a-b2af-3c734677aa2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:20.959471 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cd695-5daf-4c2a-b2af-3c734677aa2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:20.960149 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cd695-5daf-4c2a-b2af-3c734677aa2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:20.961951 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: WARNING py.warnings [None req-ad3cd695-5daf-4c2a-b2af-3c734677aa2c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:31:20.961951 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: warnings.warn(msg) May 27 03:31:20.961951 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: May 27 03:31:20.963187 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cd695-5daf-4c2a-b2af-3c734677aa2c admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:21.083927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 750/1497] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:31:20 2022] POST /identity/v3/projects => generated 379 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:21.088225 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ac70c-2b9d-493c-a59b-3d441cef9229 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:21.098416 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ac70c-2b9d-493c-a59b-3d441cef9229 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:21.152774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7743c69-3ef7-4489-a238-480d0ce7280c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:21.153434 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7743c69-3ef7-4489-a238-480d0ce7280c admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:21.153719 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7743c69-3ef7-4489-a238-480d0ce7280c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:21.153995 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7743c69-3ef7-4489-a238-480d0ce7280c admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:21.154710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7743c69-3ef7-4489-a238-480d0ce7280c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:21.155766 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: WARNING py.warnings [None req-c7743c69-3ef7-4489-a238-480d0ce7280c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:31:21.155766 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: warnings.warn(msg) May 27 03:31:21.155766 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: May 27 03:31:21.157527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7743c69-3ef7-4489-a238-480d0ce7280c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:21.243008 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 748/1498] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:31:21 2022] POST /identity/v3/users => generated 290 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:21.247260 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cd695-5daf-4c2a-b2af-3c734677aa2c admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:21.257756 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cd695-5daf-4c2a-b2af-3c734677aa2c admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:21.262366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663cf54-6a62-420c-94c4-96bf90ff7649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:21.263066 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663cf54-6a62-420c-94c4-96bf90ff7649 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:21.263348 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663cf54-6a62-420c-94c4-96bf90ff7649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:21.263617 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663cf54-6a62-420c-94c4-96bf90ff7649 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:21.264358 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9663cf54-6a62-420c-94c4-96bf90ff7649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:21.265365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: WARNING py.warnings [None req-9663cf54-6a62-420c-94c4-96bf90ff7649 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 03:31:21.265365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: warnings.warn(msg) May 27 03:31:21.265365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: May 27 03:31:21.266240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9663cf54-6a62-420c-94c4-96bf90ff7649 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:21.278770 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 751/1499] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:21.283072 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7743c69-3ef7-4489-a238-480d0ce7280c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:21.292733 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7743c69-3ef7-4489-a238-480d0ce7280c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:21.297937 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ff47cd-a945-4939-9ae7-753d1bc8ecce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:21.298630 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff47cd-a945-4939-9ae7-753d1bc8ecce admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:21.298928 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff47cd-a945-4939-9ae7-753d1bc8ecce admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:21.299227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ff47cd-a945-4939-9ae7-753d1bc8ecce admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:21.299942 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ff47cd-a945-4939-9ae7-753d1bc8ecce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:21.300946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: WARNING py.warnings [None req-90ff47cd-a945-4939-9ae7-753d1bc8ecce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 03:31:21.300946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: warnings.warn(msg) May 27 03:31:21.300946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: May 27 03:31:21.301822 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ff47cd-a945-4939-9ae7-753d1bc8ecce admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:21.315615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 749/1500] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:21.319568 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663cf54-6a62-420c-94c4-96bf90ff7649 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:21.330118 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663cf54-6a62-420c-94c4-96bf90ff7649 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:21.334475 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a8b3c-4f87-4f2b-b287-44901f58c387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:21.335371 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a8b3c-4f87-4f2b-b287-44901f58c387 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:21.335650 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a8b3c-4f87-4f2b-b287-44901f58c387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:21.335920 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a8b3c-4f87-4f2b-b287-44901f58c387 admin admin] PATH_INFO: `/v3/projects/dcaa6a38ac84497daf8344247e53f0d3/users/30f4d64209cc49459a1e831194743b3b/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:21.350330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a8b3c-4f87-4f2b-b287-44901f58c387 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcaa6a38ac84497daf8344247e53f0d3, user_id=30f4d64209cc49459a1e831194743b3b, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:21.351393 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: WARNING py.warnings [None req-812a8b3c-4f87-4f2b-b287-44901f58c387 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:31:21.351393 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: warnings.warn(msg) May 27 03:31:21.351393 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: May 27 03:31:21.353296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a8b3c-4f87-4f2b-b287-44901f58c387 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:21.413816 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 752/1501] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:31:21 2022] PUT /identity/v3/projects/dcaa6a38ac84497daf8344247e53f0d3/users/30f4d64209cc49459a1e831194743b3b/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:21.425021 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6c46a-f1b4-4c0d-a76f-90a51409175e None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:21.425930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6c46a-f1b4-4c0d-a76f-90a51409175e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:21.425930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6c46a-f1b4-4c0d-a76f-90a51409175e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:21.452306 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-72d6c46a-f1b4-4c0d-a76f-90a51409175e None None] MFA Rules not processed for user `30f4d64209cc49459a1e831194743b3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:21.454150 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-72d6c46a-f1b4-4c0d-a76f-90a51409175e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:21.521703 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 750/1502] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:31:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:21.535439 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe54ee-934f-481c-af1d-b8e331d37ff1 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:21.535730 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe54ee-934f-481c-af1d-b8e331d37ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:21.536012 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe54ee-934f-481c-af1d-b8e331d37ff1 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:21.537375 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 753/1503] 10.222.0.24 () {54 vars in 963 bytes} [Fri May 27 03:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:31:21.542337 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ace777-bb2a-47ce-8e48-a757c91c276c None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:21.546742 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ace777-bb2a-47ce-8e48-a757c91c276c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:21.546742 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ace777-bb2a-47ce-8e48-a757c91c276c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:21.561860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-00ace777-bb2a-47ce-8e48-a757c91c276c None None] MFA Rules not processed for user `c81ae2718faf4e59b799add42fb41c4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:21.563754 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-00ace777-bb2a-47ce-8e48-a757c91c276c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:21.606729 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:21.614416 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 751/1504] 10.222.0.24 () {58 vars in 1060 bytes} [Fri May 27 03:31:21 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:21.622127 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe54ee-934f-481c-af1d-b8e331d37ff1 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:21.633287 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe54ee-934f-481c-af1d-b8e331d37ff1 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:21.636742 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea20d3-0df3-49e1-aae1-baa36e977c37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:21.637374 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea20d3-0df3-49e1-aae1-baa36e977c37 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:21.637656 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea20d3-0df3-49e1-aae1-baa36e977c37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:21.637933 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea20d3-0df3-49e1-aae1-baa36e977c37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:21.646026 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea20d3-0df3-49e1-aae1-baa36e977c37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:21.647519 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea20d3-0df3-49e1-aae1-baa36e977c37 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:21.656289 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 754/1505] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:31:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:21.704318 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29689f5b-a08d-4b7c-9b4b-c73ff99a2852 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:21.709759 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29689f5b-a08d-4b7c-9b4b-c73ff99a2852 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:21.709759 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29689f5b-a08d-4b7c-9b4b-c73ff99a2852 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:21.709759 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 752/1506] 10.222.0.24 () {52 vars in 829 bytes} [Fri May 27 03:31:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:31:21.710043 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea20d3-0df3-49e1-aae1-baa36e977c37 service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:21.769609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea20d3-0df3-49e1-aae1-baa36e977c37 service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:21.774143 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a0cc0-a89e-44ff-bc97-04b12b1e9b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:21.775035 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a0cc0-a89e-44ff-bc97-04b12b1e9b9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:21.775313 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a0cc0-a89e-44ff-bc97-04b12b1e9b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:21.775585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a0cc0-a89e-44ff-bc97-04b12b1e9b9c admin admin] PATH_INFO: `/v3/users/30f4d64209cc49459a1e831194743b3b` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:21.776744 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a0cc0-a89e-44ff-bc97-04b12b1e9b9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=30f4d64209cc49459a1e831194743b3b)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:21.777753 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: WARNING py.warnings [None req-3f6a0cc0-a89e-44ff-bc97-04b12b1e9b9c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:31:21.777753 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: warnings.warn(msg) May 27 03:31:21.777753 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: May 27 03:31:21.779057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a0cc0-a89e-44ff-bc97-04b12b1e9b9c admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:21.974781 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-3f6a0cc0-a89e-44ff-bc97-04b12b1e9b9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30f4d64209cc49459a1e831194743b3b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:22.040490 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-3f6a0cc0-a89e-44ff-bc97-04b12b1e9b9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30f4d64209cc49459a1e831194743b3b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:22.041110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-3f6a0cc0-a89e-44ff-bc97-04b12b1e9b9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30f4d64209cc49459a1e831194743b3b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:22.053637 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-3f6a0cc0-a89e-44ff-bc97-04b12b1e9b9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30f4d64209cc49459a1e831194743b3b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:22.055992 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-3f6a0cc0-a89e-44ff-bc97-04b12b1e9b9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30f4d64209cc49459a1e831194743b3b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:22.070613 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:22.079329 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 755/1507] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:31:21 2022] DELETE /identity/v3/users/30f4d64209cc49459a1e831194743b3b => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:22.083793 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7ce2b-e414-406d-a588-33bb6c981e6c None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:22.084206 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7ce2b-e414-406d-a588-33bb6c981e6c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:22.084354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7ce2b-e414-406d-a588-33bb6c981e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:22.102884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-afc7ce2b-e414-406d-a588-33bb6c981e6c None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:22.104063 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-afc7ce2b-e414-406d-a588-33bb6c981e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:22.161097 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 753/1508] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:31:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:22.170524 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a0cc0-a89e-44ff-bc97-04b12b1e9b9c admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:22.172037 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3f6a0cc0-a89e-44ff-bc97-04b12b1e9b9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:22.229513 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a0cc0-a89e-44ff-bc97-04b12b1e9b9c admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:22.233018 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52977852-f5f9-4617-bcdb-f3ec91c469bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:22.233652 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52977852-f5f9-4617-bcdb-f3ec91c469bc service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:22.233935 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52977852-f5f9-4617-bcdb-f3ec91c469bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:22.234209 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52977852-f5f9-4617-bcdb-f3ec91c469bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:22.243433 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52977852-f5f9-4617-bcdb-f3ec91c469bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:22.244865 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52977852-f5f9-4617-bcdb-f3ec91c469bc service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:22.254044 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 756/1509] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:31:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:22.711724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7ce2b-e414-406d-a588-33bb6c981e6c None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:22.714015 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-afc7ce2b-e414-406d-a588-33bb6c981e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:22.725076 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7ce2b-e414-406d-a588-33bb6c981e6c None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:22.730925 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7c445-d6e0-41ec-a656-e7a91b3d7f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:22.731830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7c445-d6e0-41ec-a656-e7a91b3d7f92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:22.732113 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7c445-d6e0-41ec-a656-e7a91b3d7f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:22.732387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7c445-d6e0-41ec-a656-e7a91b3d7f92 admin admin] PATH_INFO: `/v3/projects/dcaa6a38ac84497daf8344247e53f0d3` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:22.733593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7c445-d6e0-41ec-a656-e7a91b3d7f92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcaa6a38ac84497daf8344247e53f0d3)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:22.734607 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: WARNING py.warnings [None req-72d7c445-d6e0-41ec-a656-e7a91b3d7f92 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:31:22.734607 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: warnings.warn(msg) May 27 03:31:22.734607 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: May 27 03:31:22.735837 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7c445-d6e0-41ec-a656-e7a91b3d7f92 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:22.867649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-72d7c445-d6e0-41ec-a656-e7a91b3d7f92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcaa6a38ac84497daf8344247e53f0d3', 'request_id': 'req-72d7c445-d6e0-41ec-a656-e7a91b3d7f92'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:22.939250 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-72d7c445-d6e0-41ec-a656-e7a91b3d7f92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcaa6a38ac84497daf8344247e53f0d3', 'request_id': 'req-72d7c445-d6e0-41ec-a656-e7a91b3d7f92'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:22.945415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-72d7c445-d6e0-41ec-a656-e7a91b3d7f92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcaa6a38ac84497daf8344247e53f0d3', 'request_id': 'req-72d7c445-d6e0-41ec-a656-e7a91b3d7f92'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:22.964338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-72d7c445-d6e0-41ec-a656-e7a91b3d7f92 admin admin] The token cache is being invalidate because project dcaa6a38ac84497daf8344247e53f0d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:22.964624 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-72d7c445-d6e0-41ec-a656-e7a91b3d7f92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:22.965356 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-72d7c445-d6e0-41ec-a656-e7a91b3d7f92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:22.967147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 754/1510] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:31:22 2022] DELETE /identity/v3/projects/dcaa6a38ac84497daf8344247e53f0d3 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:32.250371 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:32.961358 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:06.519160 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eaf7e4-3fbf-4c72-b0cb-6fa3872c76fd None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:06.519729 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eaf7e4-3fbf-4c72-b0cb-6fa3872c76fd None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:06.519729 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eaf7e4-3fbf-4c72-b0cb-6fa3872c76fd None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:06.521047 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:33:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:33:06.526063 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f507550-1878-4122-bcbd-7eff84528d42 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:06.526516 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f507550-1878-4122-bcbd-7eff84528d42 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:06.526664 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f507550-1878-4122-bcbd-7eff84528d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:06.545052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-3f507550-1878-4122-bcbd-7eff84528d42 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:33:06.546245 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3f507550-1878-4122-bcbd-7eff84528d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:06.603883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:33:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:33:06.771800 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eaf7e4-3fbf-4c72-b0cb-6fa3872c76fd None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:06.773610 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-14eaf7e4-3fbf-4c72-b0cb-6fa3872c76fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:06.972143 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eaf7e4-3fbf-4c72-b0cb-6fa3872c76fd None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:06.977518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c832479-db48-4471-820e-b7f8a277443a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:06.978153 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c832479-db48-4471-820e-b7f8a277443a service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:06.978432 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c832479-db48-4471-820e-b7f8a277443a service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:06.978706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c832479-db48-4471-820e-b7f8a277443a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:06.987209 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c832479-db48-4471-820e-b7f8a277443a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:06.988804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c832479-db48-4471-820e-b7f8a277443a service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:06.998010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 758/1513] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:33:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:33:07.868237 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6adcb-7134-42cf-8451-13f1abeaea4d None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:07.868813 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6adcb-7134-42cf-8451-13f1abeaea4d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:07.868813 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6adcb-7134-42cf-8451-13f1abeaea4d None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:07.870216 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 756/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:33:07.875099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22115727-4a4c-42ba-8755-7db9c4c814a2 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:07.875529 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22115727-4a4c-42ba-8755-7db9c4c814a2 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:07.875641 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22115727-4a4c-42ba-8755-7db9c4c814a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:07.893137 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-22115727-4a4c-42ba-8755-7db9c4c814a2 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:33:07.894310 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-22115727-4a4c-42ba-8755-7db9c4c814a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:07.902455 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 759/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:33:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:33:08.050692 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6adcb-7134-42cf-8451-13f1abeaea4d None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:08.065526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6adcb-7134-42cf-8451-13f1abeaea4d None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:08.070690 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24a4f7-4b50-41a9-ac36-5485203553be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:08.071347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24a4f7-4b50-41a9-ac36-5485203553be service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:08.071627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24a4f7-4b50-41a9-ac36-5485203553be service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:08.071900 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24a4f7-4b50-41a9-ac36-5485203553be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:08.080410 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d24a4f7-4b50-41a9-ac36-5485203553be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:08.082003 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d24a4f7-4b50-41a9-ac36-5485203553be service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:08.091319 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 757/1516] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:33:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:33:08.373926 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498039cd-1607-4117-b6bb-eabc5e3a2ced None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:08.374144 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498039cd-1607-4117-b6bb-eabc5e3a2ced None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:08.374393 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498039cd-1607-4117-b6bb-eabc5e3a2ced None None] PATH_INFO: `/v3` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:08.375722 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 760/1517] 10.222.0.69 () {54 vars in 925 bytes} [Fri May 27 03:33:08 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:33:08.382660 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541fad8-f46f-4a45-a182-f2522ce1e6bf None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:08.382986 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541fad8-f46f-4a45-a182-f2522ce1e6bf None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:08.383267 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a541fad8-f46f-4a45-a182-f2522ce1e6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:08.400708 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-a541fad8-f46f-4a45-a182-f2522ce1e6bf None None] MFA Rules not processed for user `95f2d0aceef4451a83ee89fac5130a0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:33:08.402386 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a541fad8-f46f-4a45-a182-f2522ce1e6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:08.462040 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 758/1518] 10.222.0.69 () {58 vars in 1015 bytes} [Fri May 27 03:33:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:33:17.888242 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=37 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:18.086367 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6f103-9ff6-4ddc-8297-0117ff595ea1 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:18.086701 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6f103-9ff6-4ddc-8297-0117ff595ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:18.087001 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6f103-9ff6-4ddc-8297-0117ff595ea1 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:18.089028 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 761/1519] 10.222.0.24 () {54 vars in 960 bytes} [Fri May 27 03:33:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:33:18.094168 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c376e56-1e54-4eca-8df7-7df4ee63e8c4 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:18.094654 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c376e56-1e54-4eca-8df7-7df4ee63e8c4 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:18.094787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c376e56-1e54-4eca-8df7-7df4ee63e8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:18.112476 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-7c376e56-1e54-4eca-8df7-7df4ee63e8c4 None None] MFA Rules not processed for user `95f2d0aceef4451a83ee89fac5130a0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:33:18.113661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-7c376e56-1e54-4eca-8df7-7df4ee63e8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:18.122780 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 759/1520] 10.222.0.24 () {58 vars in 1057 bytes} [Fri May 27 03:33:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:33:18.130441 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6f103-9ff6-4ddc-8297-0117ff595ea1 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:18.144228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6f103-9ff6-4ddc-8297-0117ff595ea1 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:18.148965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cdd03b-6822-4717-988d-2063b65c0bd5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f2d0aceef4451a83ee89fac5130a0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:18.149701 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cdd03b-6822-4717-988d-2063b65c0bd5 service placement] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:18.149925 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cdd03b-6822-4717-988d-2063b65c0bd5 service placement] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:18.150191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cdd03b-6822-4717-988d-2063b65c0bd5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:18.158159 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cdd03b-6822-4717-988d-2063b65c0bd5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:18.162570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cdd03b-6822-4717-988d-2063b65c0bd5 service placement] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:18.169132 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 762/1521] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:33:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:33:19.356550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ed617-571a-4a36-a4e4-b9fe7e3644a7 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:19.357123 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ed617-571a-4a36-a4e4-b9fe7e3644a7 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:19.357123 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ed617-571a-4a36-a4e4-b9fe7e3644a7 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:19.358447 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:33:19.363544 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367221a4-648f-4d14-9006-955748aa085a None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:19.363958 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367221a4-648f-4d14-9006-955748aa085a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:19.364091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367221a4-648f-4d14-9006-955748aa085a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:19.384170 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-367221a4-648f-4d14-9006-955748aa085a None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:33:19.385858 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-367221a4-648f-4d14-9006-955748aa085a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:19.393803 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:33:19.547850 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ed617-571a-4a36-a4e4-b9fe7e3644a7 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:19.561781 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ed617-571a-4a36-a4e4-b9fe7e3644a7 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:19.567056 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe4ac05-9072-441a-99b9-e95e3bbc3678 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:19.567681 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4ac05-9072-441a-99b9-e95e3bbc3678 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:19.567957 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4ac05-9072-441a-99b9-e95e3bbc3678 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:19.568227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4ac05-9072-441a-99b9-e95e3bbc3678 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:19.577214 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe4ac05-9072-441a-99b9-e95e3bbc3678 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:19.578799 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe4ac05-9072-441a-99b9-e95e3bbc3678 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:19.588008 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 761/1524] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:33:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:33:20.445134 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5b1a3-4f63-4a02-9fda-f82645bb1b74 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:20.445669 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5b1a3-4f63-4a02-9fda-f82645bb1b74 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:20.445669 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5b1a3-4f63-4a02-9fda-f82645bb1b74 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:20.447028 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:33:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:33:20.452207 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05d870-eacd-4045-8c9d-ea13f6ca181a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:20.452620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05d870-eacd-4045-8c9d-ea13f6ca181a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:20.452803 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05d870-eacd-4045-8c9d-ea13f6ca181a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:20.471300 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-2d05d870-eacd-4045-8c9d-ea13f6ca181a None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:33:20.472493 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-2d05d870-eacd-4045-8c9d-ea13f6ca181a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:20.480873 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:33:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:33:20.630595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5b1a3-4f63-4a02-9fda-f82645bb1b74 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:20.644792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5b1a3-4f63-4a02-9fda-f82645bb1b74 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:20.650053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf19d5-4104-4245-9322-e4490cb15333 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:20.651220 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf19d5-4104-4245-9322-e4490cb15333 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:20.651494 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf19d5-4104-4245-9322-e4490cb15333 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:20.651762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf19d5-4104-4245-9322-e4490cb15333 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:20.659585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cf19d5-4104-4245-9322-e4490cb15333 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:20.661141 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cf19d5-4104-4245-9322-e4490cb15333 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:20.671071 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 765/1527] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:33:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:33:24.702306 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee5355-8a27-4a5a-b11b-4cbc0186a31b None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:24.703002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee5355-8a27-4a5a-b11b-4cbc0186a31b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:24.703002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ee5355-8a27-4a5a-b11b-4cbc0186a31b None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:24.704365 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:33:24.709567 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f2a25-b886-4fe5-a9de-9c0690318e29 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:24.709999 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f2a25-b886-4fe5-a9de-9c0690318e29 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:24.710110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f2a25-b886-4fe5-a9de-9c0690318e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:24.728082 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-163f2a25-b886-4fe5-a9de-9c0690318e29 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:33:24.729296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-163f2a25-b886-4fe5-a9de-9c0690318e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:24.736971 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:33:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:33:24.772465 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee5355-8a27-4a5a-b11b-4cbc0186a31b None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:24.774159 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c8ee5355-8a27-4a5a-b11b-4cbc0186a31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:24.830013 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ee5355-8a27-4a5a-b11b-4cbc0186a31b None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:24.833463 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb28a1a-8648-47c0-b5e4-0d03777c0b14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:24.834090 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb28a1a-8648-47c0-b5e4-0d03777c0b14 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:24.834368 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb28a1a-8648-47c0-b5e4-0d03777c0b14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:24.834640 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb28a1a-8648-47c0-b5e4-0d03777c0b14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:24.843316 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb28a1a-8648-47c0-b5e4-0d03777c0b14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:24.844740 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb28a1a-8648-47c0-b5e4-0d03777c0b14 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:24.849468 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:24.854821 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 764/1530] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:33:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:33:27.044406 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d3415-2ca6-4729-b7e8-72c4dc1cb7fa None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:27.044967 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d3415-2ca6-4729-b7e8-72c4dc1cb7fa None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:27.044967 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d3415-2ca6-4729-b7e8-72c4dc1cb7fa None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:27.046345 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:33:27.051507 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e119b-1225-410a-add0-a3d39bb37047 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:27.051933 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e119b-1225-410a-add0-a3d39bb37047 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:27.052039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e119b-1225-410a-add0-a3d39bb37047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:27.071847 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-7a4e119b-1225-410a-add0-a3d39bb37047 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:33:27.073628 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-7a4e119b-1225-410a-add0-a3d39bb37047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:27.082488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:33:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:33:27.117532 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d3415-2ca6-4729-b7e8-72c4dc1cb7fa None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:27.128695 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d3415-2ca6-4729-b7e8-72c4dc1cb7fa None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:27.132171 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86f17a5-0e7b-4ce0-8566-2746e2e59d57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:27.132804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f17a5-0e7b-4ce0-8566-2746e2e59d57 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:27.133085 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f17a5-0e7b-4ce0-8566-2746e2e59d57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:27.133365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f17a5-0e7b-4ce0-8566-2746e2e59d57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:27.142695 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86f17a5-0e7b-4ce0-8566-2746e2e59d57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:27.144163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86f17a5-0e7b-4ce0-8566-2746e2e59d57 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:27.153530 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 768/1533] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:33:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:33:30.784744 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560aaae-8318-44c8-9a40-bbb50d6fb9cb None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:30.785319 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560aaae-8318-44c8-9a40-bbb50d6fb9cb None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:30.785319 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560aaae-8318-44c8-9a40-bbb50d6fb9cb None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:30.786661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:33:30.792387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068209b2-8c61-4122-b73d-9efe3815a276 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:30.792816 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068209b2-8c61-4122-b73d-9efe3815a276 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:30.792928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068209b2-8c61-4122-b73d-9efe3815a276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:30.812731 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-068209b2-8c61-4122-b73d-9efe3815a276 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:33:30.813937 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-068209b2-8c61-4122-b73d-9efe3815a276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:30.821811 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:33:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:33:31.094021 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560aaae-8318-44c8-9a40-bbb50d6fb9cb None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:31.095823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c560aaae-8318-44c8-9a40-bbb50d6fb9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:31.279270 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560aaae-8318-44c8-9a40-bbb50d6fb9cb None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:31.283643 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa98130e-0909-4b9a-8efa-8ee1600a7a80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:31.284286 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98130e-0909-4b9a-8efa-8ee1600a7a80 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:31.284563 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98130e-0909-4b9a-8efa-8ee1600a7a80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:31.284832 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa98130e-0909-4b9a-8efa-8ee1600a7a80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:31.294582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa98130e-0909-4b9a-8efa-8ee1600a7a80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:31.296152 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa98130e-0909-4b9a-8efa-8ee1600a7a80 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:31.305720 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 767/1536] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:33:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:33:33.536033 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfddb3-819a-495e-864d-cea8de77b153 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:33.536575 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfddb3-819a-495e-864d-cea8de77b153 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:33.536575 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfddb3-819a-495e-864d-cea8de77b153 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:33.537940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:33:33.543663 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce48006-5d7b-43c6-a040-12be1c8554ac None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:33.544044 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce48006-5d7b-43c6-a040-12be1c8554ac None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:33.544213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce48006-5d7b-43c6-a040-12be1c8554ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:33.562203 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-fce48006-5d7b-43c6-a040-12be1c8554ac None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:33:33.563414 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-fce48006-5d7b-43c6-a040-12be1c8554ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:33.571053 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:33:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:33:33.843454 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfddb3-819a-495e-864d-cea8de77b153 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:33.845123 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-cbcfddb3-819a-495e-864d-cea8de77b153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:33.859756 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfddb3-819a-495e-864d-cea8de77b153 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:33.863298 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5885ad7-31cb-49b8-b9ab-b0c14a3f4172 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:33.863954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5885ad7-31cb-49b8-b9ab-b0c14a3f4172 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:33.864233 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5885ad7-31cb-49b8-b9ab-b0c14a3f4172 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:33.864503 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5885ad7-31cb-49b8-b9ab-b0c14a3f4172 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:33.874158 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5885ad7-31cb-49b8-b9ab-b0c14a3f4172 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:33.875670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5885ad7-31cb-49b8-b9ab-b0c14a3f4172 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:33.884942 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 771/1539] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:33:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:33:43.557229 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=32 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:43.880884 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=33 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:03.183459 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa319c8-09c5-4282-8e38-6a9a6c9b7508 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:03.184080 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa319c8-09c5-4282-8e38-6a9a6c9b7508 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:03.184080 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa319c8-09c5-4282-8e38-6a9a6c9b7508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:03.213055 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-daa319c8-09c5-4282-8e38-6a9a6c9b7508 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:03.214708 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-daa319c8-09c5-4282-8e38-6a9a6c9b7508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:03.228474 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 769/1540] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:03.238167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f87b1d-12c2-45af-bd4b-52fce551afaa None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:03.238784 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f87b1d-12c2-45af-bd4b-52fce551afaa None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:03.238987 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f87b1d-12c2-45af-bd4b-52fce551afaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:03.268033 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-74f87b1d-12c2-45af-bd4b-52fce551afaa None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:03.269574 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-74f87b1d-12c2-45af-bd4b-52fce551afaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:03.279744 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 772/1541] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:03.284022 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa319c8-09c5-4282-8e38-6a9a6c9b7508 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:03.297346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa319c8-09c5-4282-8e38-6a9a6c9b7508 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:03.307621 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9a45c-41d6-4e03-ae95-a6edef62acae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:03.308589 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9a45c-41d6-4e03-ae95-a6edef62acae admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:03.309039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9a45c-41d6-4e03-ae95-a6edef62acae admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:03.309493 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9a45c-41d6-4e03-ae95-a6edef62acae admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:03.310404 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef9a45c-41d6-4e03-ae95-a6edef62acae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:35:03.310894 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef9a45c-41d6-4e03-ae95-a6edef62acae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:03.313430 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef9a45c-41d6-4e03-ae95-a6edef62acae admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:03.328445 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 770/1542] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:35:03.336765 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f87b1d-12c2-45af-bd4b-52fce551afaa None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:03.347544 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f87b1d-12c2-45af-bd4b-52fce551afaa None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:03.355788 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4245171-d70d-4501-8d64-12b6dab32c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:03.356689 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4245171-d70d-4501-8d64-12b6dab32c09 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:03.357138 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4245171-d70d-4501-8d64-12b6dab32c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:03.357581 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4245171-d70d-4501-8d64-12b6dab32c09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:03.358442 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4245171-d70d-4501-8d64-12b6dab32c09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:03.360552 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4245171-d70d-4501-8d64-12b6dab32c09 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:03.493117 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e81c14-0bdc-475a-ad07-e0eae85aab6f None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:03.493683 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e81c14-0bdc-475a-ad07-e0eae85aab6f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:03.494141 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e81c14-0bdc-475a-ad07-e0eae85aab6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:03.508488 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 773/1543] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:03 2022] POST /identity/v3/projects => generated 373 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:03.520646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-a4e81c14-0bdc-475a-ad07-e0eae85aab6f None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:03.521105 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4245171-d70d-4501-8d64-12b6dab32c09 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:03.521942 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a4e81c14-0bdc-475a-ad07-e0eae85aab6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:03.651622 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 771/1544] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) May 27 03:35:03.653171 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4904293-1a00-49c0-9efe-bb0dfbdae34b None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:03.653624 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4904293-1a00-49c0-9efe-bb0dfbdae34b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:03.654175 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4904293-1a00-49c0-9efe-bb0dfbdae34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:03.664501 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4245171-d70d-4501-8d64-12b6dab32c09 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:03.671247 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38803e-10a0-48c1-baf2-792d1a758b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:03.672188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38803e-10a0-48c1-baf2-792d1a758b77 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:03.672672 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38803e-10a0-48c1-baf2-792d1a758b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:03.673147 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38803e-10a0-48c1-baf2-792d1a758b77 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:03.674064 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38803e-10a0-48c1-baf2-792d1a758b77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:03.675328 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: WARNING py.warnings [None req-ef38803e-10a0-48c1-baf2-792d1a758b77 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:35:03.675328 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: warnings.warn(msg) May 27 03:35:03.675328 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: May 27 03:35:03.676543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38803e-10a0-48c1-baf2-792d1a758b77 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:03.676679 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-b4904293-1a00-49c0-9efe-bb0dfbdae34b None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:03.677813 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b4904293-1a00-49c0-9efe-bb0dfbdae34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:03.686548 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 772/1545] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:03.687966 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e81f1-2235-483a-b9ee-2f4ed9862c8f None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:03.688150 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e81f1-2235-483a-b9ee-2f4ed9862c8f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:03.688400 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e81f1-2235-483a-b9ee-2f4ed9862c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:03.706369 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-fe7e81f1-2235-483a-b9ee-2f4ed9862c8f None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:03.707568 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-fe7e81f1-2235-483a-b9ee-2f4ed9862c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:03.715593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 773/1546] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:03.716804 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b934b-fb33-447f-a305-2b119c827d68 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:03.717080 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b934b-fb33-447f-a305-2b119c827d68 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:03.717349 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b934b-fb33-447f-a305-2b119c827d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:03.735102 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-c62b934b-fb33-447f-a305-2b119c827d68 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:03.736249 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c62b934b-fb33-447f-a305-2b119c827d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:03.744559 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 774/1547] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:03.745754 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c51de3-15b6-4c9b-b54a-035cfe109a10 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:03.747972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c51de3-15b6-4c9b-b54a-035cfe109a10 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:03.747972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c51de3-15b6-4c9b-b54a-035cfe109a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:03.763604 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-64c51de3-15b6-4c9b-b54a-035cfe109a10 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:03.764787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-64c51de3-15b6-4c9b-b54a-035cfe109a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:03.772494 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 775/1548] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:03.772936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 774/1549] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:03 2022] POST /identity/v3/users => generated 287 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:03.773689 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e63911-3ba7-4914-93a6-f236a3ce3867 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:03.773971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e63911-3ba7-4914-93a6-f236a3ce3867 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:03.774371 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e63911-3ba7-4914-93a6-f236a3ce3867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:03.774493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cc6ed-4cff-4e97-9d2f-2312ac2e535f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:03.774671 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cc6ed-4cff-4e97-9d2f-2312ac2e535f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:03.774947 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cc6ed-4cff-4e97-9d2f-2312ac2e535f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:03.796721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-a0e63911-3ba7-4914-93a6-f236a3ce3867 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:03.797490 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-539cc6ed-4cff-4e97-9d2f-2312ac2e535f None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:03.797922 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a0e63911-3ba7-4914-93a6-f236a3ce3867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:03.798668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-539cc6ed-4cff-4e97-9d2f-2312ac2e535f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:03.806253 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 776/1550] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:03.810677 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 775/1551] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:03.810887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9685c-6657-44a8-94d7-a1e2ecea574d None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:03.810887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9685c-6657-44a8-94d7-a1e2ecea574d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:03.810887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9685c-6657-44a8-94d7-a1e2ecea574d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:03.811169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5edc2-e4ca-435a-90ce-ed796cd6ab0f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:03.811169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5edc2-e4ca-435a-90ce-ed796cd6ab0f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:03.811169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5edc2-e4ca-435a-90ce-ed796cd6ab0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:03.829395 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-8bf9685c-6657-44a8-94d7-a1e2ecea574d None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:03.830554 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-06f5edc2-e4ca-435a-90ce-ed796cd6ab0f None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:03.830696 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8bf9685c-6657-44a8-94d7-a1e2ecea574d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:03.832138 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-06f5edc2-e4ca-435a-90ce-ed796cd6ab0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:03.840683 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 777/1552] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:03.841913 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0585019-b173-44e2-a5a9-d5001e1ac3a3 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:03.842201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0585019-b173-44e2-a5a9-d5001e1ac3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:03.843128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0585019-b173-44e2-a5a9-d5001e1ac3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:03.851976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 776/1553] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:03.851976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058c11d-0123-4eb7-8709-a9311f0c96d8 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:03.851976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058c11d-0123-4eb7-8709-a9311f0c96d8 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:03.851976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058c11d-0123-4eb7-8709-a9311f0c96d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:03.868317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-e0585019-b173-44e2-a5a9-d5001e1ac3a3 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:03.869250 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-2058c11d-0123-4eb7-8709-a9311f0c96d8 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:03.869498 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e0585019-b173-44e2-a5a9-d5001e1ac3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:03.870415 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-2058c11d-0123-4eb7-8709-a9311f0c96d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:03.877215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 778/1554] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:03.877974 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0585019-b173-44e2-a5a9-d5001e1ac3a3 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:03.882625 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 777/1555] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:03.884060 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534ba4d-0979-4d66-8822-fc357831b2f0 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:03.884352 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534ba4d-0979-4d66-8822-fc357831b2f0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:03.884627 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534ba4d-0979-4d66-8822-fc357831b2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:03.887204 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0585019-b173-44e2-a5a9-d5001e1ac3a3 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:03.892641 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d401b7-df0c-4683-b283-50b92a39be1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:03.893333 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d401b7-df0c-4683-b283-50b92a39be1d admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:03.893616 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d401b7-df0c-4683-b283-50b92a39be1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:03.893895 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d401b7-df0c-4683-b283-50b92a39be1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:03.894872 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d401b7-df0c-4683-b283-50b92a39be1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:03.896534 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d401b7-df0c-4683-b283-50b92a39be1d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:03.903840 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-c534ba4d-0979-4d66-8822-fc357831b2f0 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:03.904964 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c534ba4d-0979-4d66-8822-fc357831b2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:03.907192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 779/1556] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:03.908064 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d401b7-df0c-4683-b283-50b92a39be1d admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:03.913637 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 778/1557] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:03.915947 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c36e4-9ea4-4bf2-84ea-0e917c5ab8eb None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:03.916355 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c36e4-9ea4-4bf2-84ea-0e917c5ab8eb None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:03.916630 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c36e4-9ea4-4bf2-84ea-0e917c5ab8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:03.919135 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d401b7-df0c-4683-b283-50b92a39be1d admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:03.923610 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc9993-44b1-4ba6-a12c-cead0463a756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:03.924256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc9993-44b1-4ba6-a12c-cead0463a756 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:03.924658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc9993-44b1-4ba6-a12c-cead0463a756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:03.924941 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc9993-44b1-4ba6-a12c-cead0463a756 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:03.925675 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc9993-44b1-4ba6-a12c-cead0463a756 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:35:03.925966 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc9993-44b1-4ba6-a12c-cead0463a756 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:03.927596 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc9993-44b1-4ba6-a12c-cead0463a756 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:03.934147 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-ae3c36e4-9ea4-4bf2-84ea-0e917c5ab8eb None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:03.935351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ae3c36e4-9ea4-4bf2-84ea-0e917c5ab8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:03.939540 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 780/1558] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:35:03.940814 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c09488-2592-4349-abc9-141c2ed4a8a9 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:03.941104 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c09488-2592-4349-abc9-141c2ed4a8a9 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:03.941379 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c09488-2592-4349-abc9-141c2ed4a8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:03.945663 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 779/1559] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:03.946431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c36e4-9ea4-4bf2-84ea-0e917c5ab8eb None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:03.955696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c36e4-9ea4-4bf2-84ea-0e917c5ab8eb None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:03.961280 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065c766-09e2-4a65-95b9-637e9b6940f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:03.961903 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-82c09488-2592-4349-abc9-141c2ed4a8a9 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:03.962010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065c766-09e2-4a65-95b9-637e9b6940f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:03.962194 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065c766-09e2-4a65-95b9-637e9b6940f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:03.962467 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0065c766-09e2-4a65-95b9-637e9b6940f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:03.963133 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-82c09488-2592-4349-abc9-141c2ed4a8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:03.963250 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0065c766-09e2-4a65-95b9-637e9b6940f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:35:03.963548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0065c766-09e2-4a65-95b9-637e9b6940f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:03.964558 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: WARNING py.warnings [None req-0065c766-09e2-4a65-95b9-637e9b6940f2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 03:35:03.964558 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: warnings.warn(msg) May 27 03:35:03.964558 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: May 27 03:35:03.965579 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0065c766-09e2-4a65-95b9-637e9b6940f2 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:03.971087 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 781/1560] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:03.971863 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c09488-2592-4349-abc9-141c2ed4a8a9 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:03.980832 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 780/1561] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:35:03.981298 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c09488-2592-4349-abc9-141c2ed4a8a9 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:03.981919 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065c766-09e2-4a65-95b9-637e9b6940f2 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:03.985741 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103f330-3e29-4259-afdc-72a9ba7fba0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:03.986373 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103f330-3e29-4259-afdc-72a9ba7fba0b admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:03.986651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103f330-3e29-4259-afdc-72a9ba7fba0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:03.986948 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103f330-3e29-4259-afdc-72a9ba7fba0b admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:03.987762 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103f330-3e29-4259-afdc-72a9ba7fba0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:35:03.987960 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103f330-3e29-4259-afdc-72a9ba7fba0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:03.989537 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103f330-3e29-4259-afdc-72a9ba7fba0b admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:03.992697 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0065c766-09e2-4a65-95b9-637e9b6940f2 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:03.997403 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13398f7-ae77-464f-a090-b18db23c4389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:03.998063 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13398f7-ae77-464f-a090-b18db23c4389 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:03.998483 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13398f7-ae77-464f-a090-b18db23c4389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:03.998759 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13398f7-ae77-464f-a090-b18db23c4389 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:03.999500 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13398f7-ae77-464f-a090-b18db23c4389 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:35:03.999801 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13398f7-ae77-464f-a090-b18db23c4389 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:04.001363 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13398f7-ae77-464f-a090-b18db23c4389 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:04.001469 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 782/1562] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:35:04.002223 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103f330-3e29-4259-afdc-72a9ba7fba0b admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:04.013982 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 781/1563] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:35:04.014727 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13398f7-ae77-464f-a090-b18db23c4389 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:04.017414 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103f330-3e29-4259-afdc-72a9ba7fba0b admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:04.023477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332cfbce-86ff-430f-95ca-32f9a1fbe52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:04.024128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332cfbce-86ff-430f-95ca-32f9a1fbe52c admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:04.024415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332cfbce-86ff-430f-95ca-32f9a1fbe52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:04.024687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332cfbce-86ff-430f-95ca-32f9a1fbe52c admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:04.024805 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13398f7-ae77-464f-a090-b18db23c4389 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:04.025411 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332cfbce-86ff-430f-95ca-32f9a1fbe52c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:04.027047 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332cfbce-86ff-430f-95ca-32f9a1fbe52c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:04.029485 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b2d41-121c-4be8-a2be-1e17406f436e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:04.030114 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b2d41-121c-4be8-a2be-1e17406f436e admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:04.030397 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b2d41-121c-4be8-a2be-1e17406f436e admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:04.030673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b2d41-121c-4be8-a2be-1e17406f436e admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:04.031420 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b2d41-121c-4be8-a2be-1e17406f436e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:35:04.031729 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b2d41-121c-4be8-a2be-1e17406f436e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:04.033289 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b2d41-121c-4be8-a2be-1e17406f436e admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:04.040538 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 783/1564] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:04.040538 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332cfbce-86ff-430f-95ca-32f9a1fbe52c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:04.052974 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 782/1565] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:35:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:35:04.052974 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b2d41-121c-4be8-a2be-1e17406f436e admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:04.053232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332cfbce-86ff-430f-95ca-32f9a1fbe52c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:04.053232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6abd7b-b7d8-42a7-949b-167344bb1f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:04.053232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6abd7b-b7d8-42a7-949b-167344bb1f18 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:04.053232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6abd7b-b7d8-42a7-949b-167344bb1f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:04.053584 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6abd7b-b7d8-42a7-949b-167344bb1f18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:04.053892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6abd7b-b7d8-42a7-949b-167344bb1f18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:35:04.054201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6abd7b-b7d8-42a7-949b-167344bb1f18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:04.055783 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6abd7b-b7d8-42a7-949b-167344bb1f18 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:04.058478 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b2d41-121c-4be8-a2be-1e17406f436e admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:04.063025 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a22fad-932e-4cbd-bfc6-a905500926a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:04.063657 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a22fad-932e-4cbd-bfc6-a905500926a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:04.063936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a22fad-932e-4cbd-bfc6-a905500926a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:04.064210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a22fad-932e-4cbd-bfc6-a905500926a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:04.064897 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a22fad-932e-4cbd-bfc6-a905500926a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:04.066759 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a22fad-932e-4cbd-bfc6-a905500926a3 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:04.067569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 784/1566] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:35:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:35:04.071527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6abd7b-b7d8-42a7-949b-167344bb1f18 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:04.086306 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6abd7b-b7d8-42a7-949b-167344bb1f18 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:04.090940 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d69c0-e5d2-446c-aeb2-09109ea44349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:04.094406 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d69c0-e5d2-446c-aeb2-09109ea44349 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:04.094406 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d69c0-e5d2-446c-aeb2-09109ea44349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:04.094406 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d69c0-e5d2-446c-aeb2-09109ea44349 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:04.094406 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3d69c0-e5d2-446c-aeb2-09109ea44349 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:35:04.094406 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3d69c0-e5d2-446c-aeb2-09109ea44349 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:04.094813 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3d69c0-e5d2-446c-aeb2-09109ea44349 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:04.107799 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 785/1567] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:35:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:35:04.109707 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d69c0-e5d2-446c-aeb2-09109ea44349 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:04.120396 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3d69c0-e5d2-446c-aeb2-09109ea44349 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:04.124987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effdd793-ec3e-4259-ae81-1b9b1bc9d156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:04.125637 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effdd793-ec3e-4259-ae81-1b9b1bc9d156 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:04.125917 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effdd793-ec3e-4259-ae81-1b9b1bc9d156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:04.126202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effdd793-ec3e-4259-ae81-1b9b1bc9d156 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:04.126915 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effdd793-ec3e-4259-ae81-1b9b1bc9d156 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:04.127934 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: WARNING py.warnings [None req-effdd793-ec3e-4259-ae81-1b9b1bc9d156 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:35:04.127934 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: warnings.warn(msg) May 27 03:35:04.127934 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: May 27 03:35:04.129118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effdd793-ec3e-4259-ae81-1b9b1bc9d156 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:04.139778 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:04.174702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 783/1568] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:04 2022] POST /identity/v3/projects => generated 405 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:04.174702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a22fad-932e-4cbd-bfc6-a905500926a3 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:04.208413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:04.240067 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a22fad-932e-4cbd-bfc6-a905500926a3 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:04.244816 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada31b6a-fda0-4608-8939-d3930fcfc972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:04.245467 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada31b6a-fda0-4608-8939-d3930fcfc972 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:04.245748 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada31b6a-fda0-4608-8939-d3930fcfc972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:04.246023 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada31b6a-fda0-4608-8939-d3930fcfc972 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:04.246705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada31b6a-fda0-4608-8939-d3930fcfc972 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:04.248631 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada31b6a-fda0-4608-8939-d3930fcfc972 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:04.268540 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 786/1569] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:04 2022] POST /identity/v3/projects => generated 389 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:04.268540 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effdd793-ec3e-4259-ae81-1b9b1bc9d156 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:04.326898 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effdd793-ec3e-4259-ae81-1b9b1bc9d156 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:04.331385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7bfda-ad24-4bda-a9d1-be257a66973c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:04.332028 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7bfda-ad24-4bda-a9d1-be257a66973c admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:04.332307 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7bfda-ad24-4bda-a9d1-be257a66973c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:04.332578 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7bfda-ad24-4bda-a9d1-be257a66973c admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:04.333255 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7bfda-ad24-4bda-a9d1-be257a66973c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:04.335795 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7bfda-ad24-4bda-a9d1-be257a66973c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:04.362007 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 784/1570] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:04 2022] POST /identity/v3/projects => generated 369 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:04.363051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada31b6a-fda0-4608-8939-d3930fcfc972 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:04.424879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada31b6a-fda0-4608-8939-d3930fcfc972 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:04.429601 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef9f8f-be8e-498f-baa3-945c1281e15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:04.430485 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef9f8f-be8e-498f-baa3-945c1281e15b admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:04.430762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef9f8f-be8e-498f-baa3-945c1281e15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:04.431073 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef9f8f-be8e-498f-baa3-945c1281e15b admin admin] PATH_INFO: `/v3/projects/5f8d42bcfc0947baa7debb6c4eb692c7/users/1d6495f63468440991b39a43a11c856b/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:04.446166 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ef9f8f-be8e-498f-baa3-945c1281e15b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f8d42bcfc0947baa7debb6c4eb692c7, user_id=1d6495f63468440991b39a43a11c856b, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:04.448910 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 787/1571] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:04 2022] POST /identity/v3/projects => generated 371 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:04.449733 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7bfda-ad24-4bda-a9d1-be257a66973c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:04.452294 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ef9f8f-be8e-498f-baa3-945c1281e15b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:04.518754 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7bfda-ad24-4bda-a9d1-be257a66973c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:04.524765 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7c49ed-0394-442f-918f-5f5e33954d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:04.525426 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c49ed-0394-442f-918f-5f5e33954d11 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:04.525709 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c49ed-0394-442f-918f-5f5e33954d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:04.525984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c49ed-0394-442f-918f-5f5e33954d11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:04.526685 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7c49ed-0394-442f-918f-5f5e33954d11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:04.526944 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 785/1572] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:04 2022] PUT /identity/v3/projects/5f8d42bcfc0947baa7debb6c4eb692c7/users/1d6495f63468440991b39a43a11c856b/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:04.527835 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef9f8f-be8e-498f-baa3-945c1281e15b admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:04.528626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7c49ed-0394-442f-918f-5f5e33954d11 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:04.594195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef9f8f-be8e-498f-baa3-945c1281e15b admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:04.598787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97217ec-6d97-4152-ad7b-b734d131c0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:04.599505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97217ec-6d97-4152-ad7b-b734d131c0a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:04.599820 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97217ec-6d97-4152-ad7b-b734d131c0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:04.600095 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97217ec-6d97-4152-ad7b-b734d131c0a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:04.600773 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97217ec-6d97-4152-ad7b-b734d131c0a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:04.602670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97217ec-6d97-4152-ad7b-b734d131c0a5 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:04.619125 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 788/1573] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:04 2022] POST /identity/v3/projects => generated 365 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:04.619943 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7c49ed-0394-442f-918f-5f5e33954d11 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:04.681196 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7c49ed-0394-442f-918f-5f5e33954d11 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:04.685769 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d6a88-223e-4b0b-b81a-48eefe8c453e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:04.686409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d6a88-223e-4b0b-b81a-48eefe8c453e admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:04.686689 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d6a88-223e-4b0b-b81a-48eefe8c453e admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:04.686992 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d6a88-223e-4b0b-b81a-48eefe8c453e admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:04.687671 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9d6a88-223e-4b0b-b81a-48eefe8c453e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:04.689547 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9d6a88-223e-4b0b-b81a-48eefe8c453e admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:04.694156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:04.705178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 786/1574] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:04 2022] POST /identity/v3/projects => generated 385 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:04.706153 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97217ec-6d97-4152-ad7b-b734d131c0a5 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:04.726751 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:04.765746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97217ec-6d97-4152-ad7b-b734d131c0a5 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:04.770251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e0b34-c977-4ec4-a9b2-cbf54fe781cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:04.770932 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e0b34-c977-4ec4-a9b2-cbf54fe781cd admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:04.771212 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e0b34-c977-4ec4-a9b2-cbf54fe781cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:04.771508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e0b34-c977-4ec4-a9b2-cbf54fe781cd admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:04.772309 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e0b34-c977-4ec4-a9b2-cbf54fe781cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:04.777749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e0b34-c977-4ec4-a9b2-cbf54fe781cd admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:04.784610 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 789/1575] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:04 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:04.789119 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d6a88-223e-4b0b-b81a-48eefe8c453e admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:04.851460 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d6a88-223e-4b0b-b81a-48eefe8c453e admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:04.856177 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700593aa-aa83-4798-9aec-44c7e83a95f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:04.856809 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700593aa-aa83-4798-9aec-44c7e83a95f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:04.857096 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700593aa-aa83-4798-9aec-44c7e83a95f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:04.857377 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700593aa-aa83-4798-9aec-44c7e83a95f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:04.858072 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700593aa-aa83-4798-9aec-44c7e83a95f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:04.860050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700593aa-aa83-4798-9aec-44c7e83a95f8 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:04.867194 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 787/1576] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:04 2022] POST /identity/v3/users => generated 303 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:04.867194 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e0b34-c977-4ec4-a9b2-cbf54fe781cd admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:04.874534 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e0b34-c977-4ec4-a9b2-cbf54fe781cd admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:04.879717 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7006e54-630d-4f84-a23f-90ad1692c274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:04.880362 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7006e54-630d-4f84-a23f-90ad1692c274 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:04.880642 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7006e54-630d-4f84-a23f-90ad1692c274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:04.880915 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7006e54-630d-4f84-a23f-90ad1692c274 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:04.882150 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7006e54-630d-4f84-a23f-90ad1692c274 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:04.884088 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7006e54-630d-4f84-a23f-90ad1692c274 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:04.940965 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 790/1577] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:04 2022] POST /identity/v3/users => generated 295 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:04.941706 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700593aa-aa83-4798-9aec-44c7e83a95f8 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:04.954234 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700593aa-aa83-4798-9aec-44c7e83a95f8 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:04.958769 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef13f742-d7cc-4649-b568-c7635f952245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:04.959463 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13f742-d7cc-4649-b568-c7635f952245 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:04.959743 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13f742-d7cc-4649-b568-c7635f952245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:04.960018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13f742-d7cc-4649-b568-c7635f952245 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:04.960691 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef13f742-d7cc-4649-b568-c7635f952245 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:04.962674 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef13f742-d7cc-4649-b568-c7635f952245 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:04.982349 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 788/1578] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:04 2022] POST /identity/v3/users => generated 285 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:05.041470 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071d0c1-cec7-496f-9321-b0fc88411130 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:05.041470 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071d0c1-cec7-496f-9321-b0fc88411130 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:05.041470 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071d0c1-cec7-496f-9321-b0fc88411130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:05.041470 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-4071d0c1-cec7-496f-9321-b0fc88411130 None None] MFA Rules not processed for user `1d6495f63468440991b39a43a11c856b`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:05.041470 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-4071d0c1-cec7-496f-9321-b0fc88411130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:05.073000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 789/1579] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:05.075051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071d0c1-cec7-496f-9321-b0fc88411130 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:05.086482 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071d0c1-cec7-496f-9321-b0fc88411130 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:05.091123 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65dbf4f-0315-4fe0-8508-d45231480bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:05.091748 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65dbf4f-0315-4fe0-8508-d45231480bf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:05.092043 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65dbf4f-0315-4fe0-8508-d45231480bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:05.092310 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65dbf4f-0315-4fe0-8508-d45231480bf3 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:05.093028 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65dbf4f-0315-4fe0-8508-d45231480bf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:05.095032 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65dbf4f-0315-4fe0-8508-d45231480bf3 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:05.104171 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 791/1580] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:04 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:05.104171 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef13f742-d7cc-4649-b568-c7635f952245 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:05.112534 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef13f742-d7cc-4649-b568-c7635f952245 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:05.117706 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e7cf1-6d32-4dd7-b2b7-aea9d60252f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:05.118380 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e7cf1-6d32-4dd7-b2b7-aea9d60252f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:05.118668 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e7cf1-6d32-4dd7-b2b7-aea9d60252f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:05.119092 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e7cf1-6d32-4dd7-b2b7-aea9d60252f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:05.119772 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e7cf1-6d32-4dd7-b2b7-aea9d60252f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:05.121675 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e7cf1-6d32-4dd7-b2b7-aea9d60252f5 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:05.179324 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 790/1581] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:05 2022] POST /identity/v3/users => generated 283 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:05.180145 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65dbf4f-0315-4fe0-8508-d45231480bf3 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:05.189879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65dbf4f-0315-4fe0-8508-d45231480bf3 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:05.194652 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ff6c2-5243-42bf-a741-624a129d7ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:05.195300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ff6c2-5243-42bf-a741-624a129d7ad0 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:05.195578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ff6c2-5243-42bf-a741-624a129d7ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:05.195967 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81ff6c2-5243-42bf-a741-624a129d7ad0 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:05.196933 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81ff6c2-5243-42bf-a741-624a129d7ad0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:05.198903 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81ff6c2-5243-42bf-a741-624a129d7ad0 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:05.207046 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 792/1582] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:05 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:05.207046 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e7cf1-6d32-4dd7-b2b7-aea9d60252f5 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:05.213670 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e7cf1-6d32-4dd7-b2b7-aea9d60252f5 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:05.218170 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74cd2ee-8a2e-40fd-b302-9098a13cea66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:05.218883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74cd2ee-8a2e-40fd-b302-9098a13cea66 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:05.219169 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74cd2ee-8a2e-40fd-b302-9098a13cea66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:05.219439 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74cd2ee-8a2e-40fd-b302-9098a13cea66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:05.220151 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74cd2ee-8a2e-40fd-b302-9098a13cea66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:05.221734 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74cd2ee-8a2e-40fd-b302-9098a13cea66 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:05.232061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 793/1583] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:05.233017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74cd2ee-8a2e-40fd-b302-9098a13cea66 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:05.242794 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74cd2ee-8a2e-40fd-b302-9098a13cea66 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:05.247260 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d160e49-7d84-4005-a1f0-7bea313cade8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:05.247887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d160e49-7d84-4005-a1f0-7bea313cade8 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:05.248170 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d160e49-7d84-4005-a1f0-7bea313cade8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:05.248443 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d160e49-7d84-4005-a1f0-7bea313cade8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:05.249134 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d160e49-7d84-4005-a1f0-7bea313cade8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:05.250652 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d160e49-7d84-4005-a1f0-7bea313cade8 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:05.260235 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 794/1584] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:05.261124 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d160e49-7d84-4005-a1f0-7bea313cade8 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:05.270165 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d160e49-7d84-4005-a1f0-7bea313cade8 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:05.274659 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6949d035-fab6-46f8-b94e-f399b2f74810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:05.275286 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949d035-fab6-46f8-b94e-f399b2f74810 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:05.275569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949d035-fab6-46f8-b94e-f399b2f74810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:05.275844 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949d035-fab6-46f8-b94e-f399b2f74810 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:05.276551 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6949d035-fab6-46f8-b94e-f399b2f74810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:05.278058 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6949d035-fab6-46f8-b94e-f399b2f74810 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:05.289973 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 795/1585] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:05.291290 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b0567-c22b-441a-ae5c-5bc489bf9da2 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:05.291575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b0567-c22b-441a-ae5c-5bc489bf9da2 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:05.291850 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b0567-c22b-441a-ae5c-5bc489bf9da2 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:05.293468 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 796/1586] 10.222.0.24 () {54 vars in 961 bytes} [Fri May 27 03:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:35:05.294126 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b0567-c22b-441a-ae5c-5bc489bf9da2 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:05.298173 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 791/1587] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:05 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:05.298173 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ff6c2-5243-42bf-a741-624a129d7ad0 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:05.307112 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81ff6c2-5243-42bf-a741-624a129d7ad0 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:05.310193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b0567-c22b-441a-ae5c-5bc489bf9da2 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:05.311578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c357e-646f-4267-962c-5afe0a630f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:05.312214 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c357e-646f-4267-962c-5afe0a630f92 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:05.312492 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c357e-646f-4267-962c-5afe0a630f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:05.312865 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c357e-646f-4267-962c-5afe0a630f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:05.313284 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836bff44-62b2-42ff-a01d-798929ae2913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:05.313573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9c357e-646f-4267-962c-5afe0a630f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:05.313910 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bff44-62b2-42ff-a01d-798929ae2913 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:05.314193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bff44-62b2-42ff-a01d-798929ae2913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:05.314465 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bff44-62b2-42ff-a01d-798929ae2913 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:05.315208 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836bff44-62b2-42ff-a01d-798929ae2913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:05.315327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9c357e-646f-4267-962c-5afe0a630f92 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:05.316816 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836bff44-62b2-42ff-a01d-798929ae2913 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:05.329969 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 792/1588] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:05.330171 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 797/1589] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:05.330256 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c357e-646f-4267-962c-5afe0a630f92 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:05.330441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836bff44-62b2-42ff-a01d-798929ae2913 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:05.340236 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836bff44-62b2-42ff-a01d-798929ae2913 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:05.345621 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c357e-646f-4267-962c-5afe0a630f92 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:05.345737 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1f77f-bca3-46c8-9bba-c8b5d46e5111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:05.345737 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1f77f-bca3-46c8-9bba-c8b5d46e5111 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:05.345994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b094d-7116-472f-9f83-19f95d74083a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:05.346119 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1f77f-bca3-46c8-9bba-c8b5d46e5111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:05.346119 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1f77f-bca3-46c8-9bba-c8b5d46e5111 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:05.346368 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b094d-7116-472f-9f83-19f95d74083a admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:05.346599 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e1f77f-bca3-46c8-9bba-c8b5d46e5111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:05.346706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b094d-7116-472f-9f83-19f95d74083a admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:05.346983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b094d-7116-472f-9f83-19f95d74083a admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:05.347707 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b094d-7116-472f-9f83-19f95d74083a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:05.348202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e1f77f-bca3-46c8-9bba-c8b5d46e5111 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:05.349414 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b094d-7116-472f-9f83-19f95d74083a admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:05.358761 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 798/1590] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:05.359735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1f77f-bca3-46c8-9bba-c8b5d46e5111 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:05.363263 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 793/1591] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:05.364096 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b094d-7116-472f-9f83-19f95d74083a admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:05.369451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1f77f-bca3-46c8-9bba-c8b5d46e5111 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:05.373939 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76731312-d271-45aa-b68b-5f0f79a6e387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:05.374318 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b094d-7116-472f-9f83-19f95d74083a admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:05.374574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76731312-d271-45aa-b68b-5f0f79a6e387 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:05.374875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76731312-d271-45aa-b68b-5f0f79a6e387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:05.375156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76731312-d271-45aa-b68b-5f0f79a6e387 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:05.375864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76731312-d271-45aa-b68b-5f0f79a6e387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:05.377427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76731312-d271-45aa-b68b-5f0f79a6e387 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:05.378879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68697b-17e9-41c0-99f7-a0fd9b9e6583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:05.379513 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68697b-17e9-41c0-99f7-a0fd9b9e6583 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:05.379795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68697b-17e9-41c0-99f7-a0fd9b9e6583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:05.380068 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68697b-17e9-41c0-99f7-a0fd9b9e6583 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:05.380781 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68697b-17e9-41c0-99f7-a0fd9b9e6583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:05.382341 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68697b-17e9-41c0-99f7-a0fd9b9e6583 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:05.387678 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 799/1592] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:05.388945 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b12e7-3751-4209-9341-07ac0e3fef6e None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:05.389219 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b12e7-3751-4209-9341-07ac0e3fef6e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:05.389495 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b12e7-3751-4209-9341-07ac0e3fef6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:05.395006 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 794/1593] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:05.395006 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68697b-17e9-41c0-99f7-a0fd9b9e6583 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:05.402670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68697b-17e9-41c0-99f7-a0fd9b9e6583 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:05.407201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-596b12e7-3751-4209-9341-07ac0e3fef6e None None] MFA Rules not processed for user `22a9592143014f3287cd75c60ca17237`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:05.407339 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73926537-7e71-43d1-8ad5-aef27e1cfe21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:05.407835 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73926537-7e71-43d1-8ad5-aef27e1cfe21 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:05.408128 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73926537-7e71-43d1-8ad5-aef27e1cfe21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:05.408402 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73926537-7e71-43d1-8ad5-aef27e1cfe21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:05.408516 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-596b12e7-3751-4209-9341-07ac0e3fef6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:05.409109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73926537-7e71-43d1-8ad5-aef27e1cfe21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:05.410672 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73926537-7e71-43d1-8ad5-aef27e1cfe21 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:05.420910 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 795/1594] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:05.421684 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73926537-7e71-43d1-8ad5-aef27e1cfe21 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:05.426412 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:05.431857 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73926537-7e71-43d1-8ad5-aef27e1cfe21 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:05.437159 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf267d3-c0cc-47d3-bbf3-ca5f46ec4b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:05.437787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf267d3-c0cc-47d3-bbf3-ca5f46ec4b81 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:05.438071 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf267d3-c0cc-47d3-bbf3-ca5f46ec4b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:05.438360 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf267d3-c0cc-47d3-bbf3-ca5f46ec4b81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:05.439395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf267d3-c0cc-47d3-bbf3-ca5f46ec4b81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:05.439512 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:05.440655 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf267d3-c0cc-47d3-bbf3-ca5f46ec4b81 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:05.452268 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 796/1595] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:05.455961 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf267d3-c0cc-47d3-bbf3-ca5f46ec4b81 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:05.465725 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf267d3-c0cc-47d3-bbf3-ca5f46ec4b81 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:05.470229 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70727fad-73d4-4ddd-8fbb-18b2c3d7ed47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:05.471508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70727fad-73d4-4ddd-8fbb-18b2c3d7ed47 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:05.471508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70727fad-73d4-4ddd-8fbb-18b2c3d7ed47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:05.471508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70727fad-73d4-4ddd-8fbb-18b2c3d7ed47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:05.472138 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70727fad-73d4-4ddd-8fbb-18b2c3d7ed47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:05.473733 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70727fad-73d4-4ddd-8fbb-18b2c3d7ed47 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:05.476570 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 800/1596] 10.222.0.24 () {58 vars in 1058 bytes} [Fri May 27 03:35:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:05.477574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b12e7-3751-4209-9341-07ac0e3fef6e None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:05.484760 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 797/1597] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:05.485419 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70727fad-73d4-4ddd-8fbb-18b2c3d7ed47 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:05.492006 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b12e7-3751-4209-9341-07ac0e3fef6e None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:05.495351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70727fad-73d4-4ddd-8fbb-18b2c3d7ed47 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:05.497300 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca4049-7e1d-4630-a445-7da404685ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:05.498198 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca4049-7e1d-4630-a445-7da404685ce3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:05.498480 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca4049-7e1d-4630-a445-7da404685ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:05.498756 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca4049-7e1d-4630-a445-7da404685ce3 admin admin] PATH_INFO: `/v3/projects/2adb235324be4907ac4e0be5e0e901dd/users/8e5f5e4449f9455e82397a4137e3cf84/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:05.501490 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7d21e-7585-44c1-95db-bdc414ee8fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:05.506010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7d21e-7585-44c1-95db-bdc414ee8fc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:05.506010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7d21e-7585-44c1-95db-bdc414ee8fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:05.506010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7d21e-7585-44c1-95db-bdc414ee8fc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:05.506010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d7d21e-7585-44c1-95db-bdc414ee8fc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:05.507204 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d7d21e-7585-44c1-95db-bdc414ee8fc6 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:05.514059 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca4049-7e1d-4630-a445-7da404685ce3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2adb235324be4907ac4e0be5e0e901dd, user_id=8e5f5e4449f9455e82397a4137e3cf84, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:05.515145 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: WARNING py.warnings [None req-03ca4049-7e1d-4630-a445-7da404685ce3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:35:05.515145 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: warnings.warn(msg) May 27 03:35:05.515145 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: May 27 03:35:05.517113 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca4049-7e1d-4630-a445-7da404685ce3 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:05.523067 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 798/1598] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:05.524181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7d21e-7585-44c1-95db-bdc414ee8fc6 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:05.535449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7d21e-7585-44c1-95db-bdc414ee8fc6 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:05.542340 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffb480-a91c-4adc-924f-4f152fe2ab10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:05.543442 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffb480-a91c-4adc-924f-4f152fe2ab10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:05.543881 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffb480-a91c-4adc-924f-4f152fe2ab10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:05.544303 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ffb480-a91c-4adc-924f-4f152fe2ab10 admin admin] PATH_INFO: `/v3/projects/1377b9e588bd44f3b29b03728c418151/users/104a135932a144269f1f2aa7359098ed/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:05.560842 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 801/1599] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:05 2022] PUT /identity/v3/projects/2adb235324be4907ac4e0be5e0e901dd/users/8e5f5e4449f9455e82397a4137e3cf84/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:05.561663 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca4049-7e1d-4630-a445-7da404685ce3 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:05.564727 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ffb480-a91c-4adc-924f-4f152fe2ab10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1377b9e588bd44f3b29b03728c418151, user_id=104a135932a144269f1f2aa7359098ed, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:05.569539 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ffb480-a91c-4adc-924f-4f152fe2ab10 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:05.626825 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca4049-7e1d-4630-a445-7da404685ce3 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:05.631459 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b851bcf-de31-44c2-b0b8-fc5f8637f2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:05.632351 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b851bcf-de31-44c2-b0b8-fc5f8637f2a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:05.632626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b851bcf-de31-44c2-b0b8-fc5f8637f2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:05.632895 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b851bcf-de31-44c2-b0b8-fc5f8637f2a1 admin admin] PATH_INFO: `/v3/projects/40ff1c22c2bf4d8c8e180182751d0b36/users/5153a1b4ca604972812ea749922e9a0b/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:05.645991 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 799/1600] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:05 2022] PUT /identity/v3/projects/1377b9e588bd44f3b29b03728c418151/users/104a135932a144269f1f2aa7359098ed/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:05.647437 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffb480-a91c-4adc-924f-4f152fe2ab10 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:05.647918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b851bcf-de31-44c2-b0b8-fc5f8637f2a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40ff1c22c2bf4d8c8e180182751d0b36, user_id=5153a1b4ca604972812ea749922e9a0b, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:05.651120 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b851bcf-de31-44c2-b0b8-fc5f8637f2a1 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:05.707931 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ffb480-a91c-4adc-924f-4f152fe2ab10 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:05.713025 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a5104-d7e7-41e6-930f-12a8e59170fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:05.713660 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a5104-d7e7-41e6-930f-12a8e59170fe admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:05.713937 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a5104-d7e7-41e6-930f-12a8e59170fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:05.714211 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a5104-d7e7-41e6-930f-12a8e59170fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:05.714961 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a5104-d7e7-41e6-930f-12a8e59170fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:05.716546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a5104-d7e7-41e6-930f-12a8e59170fe admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:05.725881 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 800/1601] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:05.726628 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a5104-d7e7-41e6-930f-12a8e59170fe admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:05.735619 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a5104-d7e7-41e6-930f-12a8e59170fe admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:05.740129 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5271b1f-609c-42f2-813a-2e570085cd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:05.740973 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5271b1f-609c-42f2-813a-2e570085cd3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:05.741254 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5271b1f-609c-42f2-813a-2e570085cd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:05.741525 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5271b1f-609c-42f2-813a-2e570085cd3b admin admin] PATH_INFO: `/v3/projects/c1d79b4ed73547e8ac943686bc7661df/users/78120e5efd9a429ebf508a26d2b7bc6b/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:05.754774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 802/1602] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:05 2022] PUT /identity/v3/projects/40ff1c22c2bf4d8c8e180182751d0b36/users/5153a1b4ca604972812ea749922e9a0b/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:05.756692 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5271b1f-609c-42f2-813a-2e570085cd3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1d79b4ed73547e8ac943686bc7661df, user_id=78120e5efd9a429ebf508a26d2b7bc6b, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:05.759658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5271b1f-609c-42f2-813a-2e570085cd3b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:05.765957 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b851bcf-de31-44c2-b0b8-fc5f8637f2a1 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:05.849497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 801/1603] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:05 2022] PUT /identity/v3/projects/c1d79b4ed73547e8ac943686bc7661df/users/78120e5efd9a429ebf508a26d2b7bc6b/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:05.850511 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5271b1f-609c-42f2-813a-2e570085cd3b admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:05.857179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b851bcf-de31-44c2-b0b8-fc5f8637f2a1 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:05.877472 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:05.911322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5271b1f-609c-42f2-813a-2e570085cd3b admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:05.915941 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a98a564-212d-44f0-89fe-3a253b043fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:05.916871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98a564-212d-44f0-89fe-3a253b043fb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:05.917145 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98a564-212d-44f0-89fe-3a253b043fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:05.917410 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98a564-212d-44f0-89fe-3a253b043fb4 admin admin] PATH_INFO: `/v3/projects/fe3e221d39d8476a9a79989e792590e9/users/fb44774f48984906b23c799d139fdd27/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:05.923915 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947116c5-8f16-49c4-9ab0-f5407b6b7063 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:05.924570 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947116c5-8f16-49c4-9ab0-f5407b6b7063 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:05.924854 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947116c5-8f16-49c4-9ab0-f5407b6b7063 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:05.925139 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947116c5-8f16-49c4-9ab0-f5407b6b7063 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:05.931656 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a98a564-212d-44f0-89fe-3a253b043fb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe3e221d39d8476a9a79989e792590e9, user_id=fb44774f48984906b23c799d139fdd27, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:05.934377 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a98a564-212d-44f0-89fe-3a253b043fb4 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:05.980376 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947116c5-8f16-49c4-9ab0-f5407b6b7063 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:05.981880 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947116c5-8f16-49c4-9ab0-f5407b6b7063 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:05.987543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:05.990772 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 803/1604] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:05.991652 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947116c5-8f16-49c4-9ab0-f5407b6b7063 service nova] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:06.002193 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 802/1605] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:05 2022] PUT /identity/v3/projects/fe3e221d39d8476a9a79989e792590e9/users/fb44774f48984906b23c799d139fdd27/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:06.002193 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f2ec5-61b4-48d8-b749-5886cfea4167 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:06.002193 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f2ec5-61b4-48d8-b749-5886cfea4167 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:06.002193 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f2ec5-61b4-48d8-b749-5886cfea4167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:06.008978 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947116c5-8f16-49c4-9ab0-f5407b6b7063 service nova] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:06.023629 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-ad1f2ec5-61b4-48d8-b749-5886cfea4167 None None] MFA Rules not processed for user `8e5f5e4449f9455e82397a4137e3cf84`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:06.024870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ad1f2ec5-61b4-48d8-b749-5886cfea4167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:06.075497 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c39d46-fbb2-4fb7-85aa-12c29d7a3ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:06.076429 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c39d46-fbb2-4fb7-85aa-12c29d7a3ccb admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:06.076713 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c39d46-fbb2-4fb7-85aa-12c29d7a3ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:06.077006 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c39d46-fbb2-4fb7-85aa-12c29d7a3ccb admin admin] PATH_INFO: `/v3/projects/8ae1382d8f774b5db702d78e3bed622a/users/dac1130950b8457ea3a9349346437357/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:06.093566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c39d46-fbb2-4fb7-85aa-12c29d7a3ccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae1382d8f774b5db702d78e3bed622a, user_id=dac1130950b8457ea3a9349346437357, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:06.096632 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c39d46-fbb2-4fb7-85aa-12c29d7a3ccb admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:06.100871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 803/1606] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:05 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:06.103384 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe41fc-c024-41a0-b4e3-3a7fffd68ac8 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:06.103692 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe41fc-c024-41a0-b4e3-3a7fffd68ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:06.103973 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe41fc-c024-41a0-b4e3-3a7fffd68ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:06.121232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-52fe41fc-c024-41a0-b4e3-3a7fffd68ac8 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:06.122423 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-52fe41fc-c024-41a0-b4e3-3a7fffd68ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:06.130091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 804/1607] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:06.131505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe41fc-c024-41a0-b4e3-3a7fffd68ac8 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:06.143679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe41fc-c024-41a0-b4e3-3a7fffd68ac8 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:06.148199 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd3aa5f-e329-4879-b670-89c848774eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:06.149068 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3aa5f-e329-4879-b670-89c848774eaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:06.149344 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3aa5f-e329-4879-b670-89c848774eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:06.149616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3aa5f-e329-4879-b670-89c848774eaf admin admin] PATH_INFO: `/v3/projects/e3b236cd8e9049e6acdeb545f5d3382d/users/8584a3f99b0144979f69536f7cc86851/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:06.163775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd3aa5f-e329-4879-b670-89c848774eaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3b236cd8e9049e6acdeb545f5d3382d, user_id=8584a3f99b0144979f69536f7cc86851, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:06.166508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd3aa5f-e329-4879-b670-89c848774eaf admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:06.174921 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 804/1608] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:05 2022] PUT /identity/v3/projects/8ae1382d8f774b5db702d78e3bed622a/users/dac1130950b8457ea3a9349346437357/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:06.176002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ce44f-9aa9-401e-bf0c-e17af659cadb None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:06.176359 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ce44f-9aa9-401e-bf0c-e17af659cadb None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:06.176639 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ce44f-9aa9-401e-bf0c-e17af659cadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:06.203211 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-223ce44f-9aa9-401e-bf0c-e17af659cadb None None] MFA Rules not processed for user `5153a1b4ca604972812ea749922e9a0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:06.204421 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-223ce44f-9aa9-401e-bf0c-e17af659cadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:06.239093 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:06.257042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 805/1609] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:06 2022] PUT /identity/v3/projects/e3b236cd8e9049e6acdeb545f5d3382d/users/8584a3f99b0144979f69536f7cc86851/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:06.258574 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d39bb6-e9aa-4180-a1e4-c3d1784e2a02 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:06.258892 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d39bb6-e9aa-4180-a1e4-c3d1784e2a02 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:06.259157 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d39bb6-e9aa-4180-a1e4-c3d1784e2a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:06.274639 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 805/1610] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:06 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:06.278297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae190c43-7a09-44ff-9a34-70eb2c68401e None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:06.278297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae190c43-7a09-44ff-9a34-70eb2c68401e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:06.278297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae190c43-7a09-44ff-9a34-70eb2c68401e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:06.287130 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-f2d39bb6-e9aa-4180-a1e4-c3d1784e2a02 None None] MFA Rules not processed for user `78120e5efd9a429ebf508a26d2b7bc6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:06.288388 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f2d39bb6-e9aa-4180-a1e4-c3d1784e2a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:06.301882 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-ae190c43-7a09-44ff-9a34-70eb2c68401e None None] MFA Rules not processed for user `fb44774f48984906b23c799d139fdd27`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:06.306305 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ae190c43-7a09-44ff-9a34-70eb2c68401e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:06.352626 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 806/1611] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:06.353580 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d39bb6-e9aa-4180-a1e4-c3d1784e2a02 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:06.371792 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 806/1612] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:06 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:06.372599 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae190c43-7a09-44ff-9a34-70eb2c68401e None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:06.373965 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ae190c43-7a09-44ff-9a34-70eb2c68401e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:06.416379 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:06.430918 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d39bb6-e9aa-4180-a1e4-c3d1784e2a02 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:06.434440 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae190c43-7a09-44ff-9a34-70eb2c68401e None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:06.435809 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a565433-4739-4af4-90f3-6e550ba57331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:06.436468 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a565433-4739-4af4-90f3-6e550ba57331 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:06.436748 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a565433-4739-4af4-90f3-6e550ba57331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:06.437021 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a565433-4739-4af4-90f3-6e550ba57331 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:06.437709 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a565433-4739-4af4-90f3-6e550ba57331 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:06.438100 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e1f703-ff92-4473-b2e4-f79fd6346385 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:06.438744 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1f703-ff92-4473-b2e4-f79fd6346385 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:06.439089 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1f703-ff92-4473-b2e4-f79fd6346385 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:06.444102 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e1f703-ff92-4473-b2e4-f79fd6346385 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:06.444219 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a565433-4739-4af4-90f3-6e550ba57331 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:06.449201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e1f703-ff92-4473-b2e4-f79fd6346385 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:06.450732 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e1f703-ff92-4473-b2e4-f79fd6346385 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:06.460249 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 807/1613] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:06.461477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1cf7d-22e2-4004-887a-60b634e46b56 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:06.461824 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1cf7d-22e2-4004-887a-60b634e46b56 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:06.462100 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea1cf7d-22e2-4004-887a-60b634e46b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:06.479822 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-8ea1cf7d-22e2-4004-887a-60b634e46b56 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:06.481013 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8ea1cf7d-22e2-4004-887a-60b634e46b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:06.488783 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 808/1614] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:06.490217 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bbf37-7080-4bbd-8d42-f4948b788532 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:06.491030 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bbf37-7080-4bbd-8d42-f4948b788532 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:06.491305 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bbf37-7080-4bbd-8d42-f4948b788532 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:06.492571 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 809/1615] 10.222.0.24 () {54 vars in 912 bytes} [Fri May 27 03:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:35:06.493623 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361a341-9845-4a74-b763-fcc6e72ae15a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:06.493904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361a341-9845-4a74-b763-fcc6e72ae15a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:06.494185 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361a341-9845-4a74-b763-fcc6e72ae15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:06.511184 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-f361a341-9845-4a74-b763-fcc6e72ae15a None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:06.512659 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f361a341-9845-4a74-b763-fcc6e72ae15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:06.520652 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 807/1616] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:06 2022] POST /identity/v3/projects => generated 403 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:06.520652 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a565433-4739-4af4-90f3-6e550ba57331 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:06.520867 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 810/1617] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:06.521531 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361a341-9845-4a74-b763-fcc6e72ae15a None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:06.531416 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361a341-9845-4a74-b763-fcc6e72ae15a None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:06.585876 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:06.595929 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a1d38-e6a7-4ba7-9325-806a56944e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:06.596749 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a1d38-e6a7-4ba7-9325-806a56944e86 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:06.597363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a1d38-e6a7-4ba7-9325-806a56944e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:06.597717 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a1d38-e6a7-4ba7-9325-806a56944e86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:06.598556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a1d38-e6a7-4ba7-9325-806a56944e86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:06.601451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a1d38-e6a7-4ba7-9325-806a56944e86 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:06.616419 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a565433-4739-4af4-90f3-6e550ba57331 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:06.622729 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e38f0f-b4d8-43cf-afec-ef46a0d3be06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:06.623632 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e38f0f-b4d8-43cf-afec-ef46a0d3be06 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:06.624204 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e38f0f-b4d8-43cf-afec-ef46a0d3be06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:06.624495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e38f0f-b4d8-43cf-afec-ef46a0d3be06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:06.625427 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e38f0f-b4d8-43cf-afec-ef46a0d3be06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:06.627643 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e38f0f-b4d8-43cf-afec-ef46a0d3be06 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:06.759859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 811/1618] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:06 2022] POST /identity/v3/projects => generated 363 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:06.760052 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 808/1619] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:06 2022] POST /identity/v3/projects => generated 373 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:06.760164 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892eef2-f898-4c87-bba0-454fcaad6f1d None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:06.760164 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892eef2-f898-4c87-bba0-454fcaad6f1d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:06.760305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8d84f-84bc-4e5a-8536-457847174135 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:06.760400 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892eef2-f898-4c87-bba0-454fcaad6f1d None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:06.760476 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8d84f-84bc-4e5a-8536-457847174135 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:06.760476 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b8d84f-84bc-4e5a-8536-457847174135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:06.760621 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 812/1620] 10.222.0.24 () {54 vars in 961 bytes} [Fri May 27 03:35:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:35:06.764184 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892eef2-f898-4c87-bba0-454fcaad6f1d None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:06.786328 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-84b8d84f-84bc-4e5a-8536-457847174135 None None] MFA Rules not processed for user `22a9592143014f3287cd75c60ca17237`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:06.788033 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-84b8d84f-84bc-4e5a-8536-457847174135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:06.832792 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892eef2-f898-4c87-bba0-454fcaad6f1d None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:06.838046 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f00db1-ecd9-4ba6-8f08-80b4aacd83b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:06.838670 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f00db1-ecd9-4ba6-8f08-80b4aacd83b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:06.839174 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f00db1-ecd9-4ba6-8f08-80b4aacd83b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:06.839492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f00db1-ecd9-4ba6-8f08-80b4aacd83b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:06.870289 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 809/1621] 10.222.0.24 () {58 vars in 1009 bytes} [Fri May 27 03:35:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:06.871038 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8d84f-84bc-4e5a-8536-457847174135 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:06.910978 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f00db1-ecd9-4ba6-8f08-80b4aacd83b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:06.913030 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f00db1-ecd9-4ba6-8f08-80b4aacd83b0 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:06.922806 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 813/1622] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:06.923722 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f00db1-ecd9-4ba6-8f08-80b4aacd83b0 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:06.929367 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:06.932111 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f00db1-ecd9-4ba6-8f08-80b4aacd83b0 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:06.935993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a41845-f1dd-49f6-9638-a9d2e9dfc4a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:06.937294 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a41845-f1dd-49f6-9638-a9d2e9dfc4a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:06.937294 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a41845-f1dd-49f6-9638-a9d2e9dfc4a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:06.937294 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a41845-f1dd-49f6-9638-a9d2e9dfc4a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:06.941688 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b8d84f-84bc-4e5a-8536-457847174135 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:06.946866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a41845-f1dd-49f6-9638-a9d2e9dfc4a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:06.946996 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0771b-4564-40e6-9d23-9872b36119ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:06.947870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0771b-4564-40e6-9d23-9872b36119ce admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:06.951624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0771b-4564-40e6-9d23-9872b36119ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:06.952042 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a41845-f1dd-49f6-9638-a9d2e9dfc4a3 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:06.952124 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0771b-4564-40e6-9d23-9872b36119ce admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:06.952124 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0771b-4564-40e6-9d23-9872b36119ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:06.952124 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0771b-4564-40e6-9d23-9872b36119ce admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:06.960476 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 814/1623] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:06.961369 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a41845-f1dd-49f6-9638-a9d2e9dfc4a3 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:06.972479 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a41845-f1dd-49f6-9638-a9d2e9dfc4a3 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:06.978497 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4685987-236c-4831-b41e-1c009f250e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:06.980302 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4685987-236c-4831-b41e-1c009f250e45 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:06.980302 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4685987-236c-4831-b41e-1c009f250e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:06.980302 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4685987-236c-4831-b41e-1c009f250e45 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:06.980581 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4685987-236c-4831-b41e-1c009f250e45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:06.982321 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4685987-236c-4831-b41e-1c009f250e45 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:07.054270 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 815/1624] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:06 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:07.056327 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4685987-236c-4831-b41e-1c009f250e45 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:07.057800 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 810/1625] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:06 2022] POST /identity/v3/users => generated 308 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:07.061809 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3d2a1-b961-41ca-b7ee-806aaf494c5d None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:07.062109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3d2a1-b961-41ca-b7ee-806aaf494c5d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:07.062390 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3d2a1-b961-41ca-b7ee-806aaf494c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:07.079003 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4685987-236c-4831-b41e-1c009f250e45 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:07.085230 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ca90a-45f4-4ded-944d-22afb6311c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:07.085916 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ca90a-45f4-4ded-944d-22afb6311c39 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:07.086255 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ca90a-45f4-4ded-944d-22afb6311c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:07.086538 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ca90a-45f4-4ded-944d-22afb6311c39 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:07.087379 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ca90a-45f4-4ded-944d-22afb6311c39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:07.089531 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ca90a-45f4-4ded-944d-22afb6311c39 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:07.096775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-4ef3d2a1-b961-41ca-b7ee-806aaf494c5d None None] MFA Rules not processed for user `22a9592143014f3287cd75c60ca17237`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:07.098089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-4ef3d2a1-b961-41ca-b7ee-806aaf494c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:07.112456 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 811/1626] 10.222.0.24 () {58 vars in 1058 bytes} [Fri May 27 03:35:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:07.113763 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef3d2a1-b961-41ca-b7ee-806aaf494c5d None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:07.169963 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef3d2a1-b961-41ca-b7ee-806aaf494c5d None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:07.173267 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f18c1-2fd9-4bf0-b3f1-fe1c3a3a031d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:07.173918 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f18c1-2fd9-4bf0-b3f1-fe1c3a3a031d service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:07.174202 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f18c1-2fd9-4bf0-b3f1-fe1c3a3a031d service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:07.174607 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f18c1-2fd9-4bf0-b3f1-fe1c3a3a031d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:07.186706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f18c1-2fd9-4bf0-b3f1-fe1c3a3a031d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:07.188638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f18c1-2fd9-4bf0-b3f1-fe1c3a3a031d service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:07.206985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 812/1627] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:07.208524 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f18c1-2fd9-4bf0-b3f1-fe1c3a3a031d service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:07.220203 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f18c1-2fd9-4bf0-b3f1-fe1c3a3a031d service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:07.226604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca2e9d-a129-4355-9f61-01ffef18da5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:07.227066 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca2e9d-a129-4355-9f61-01ffef18da5f admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:07.230884 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca2e9d-a129-4355-9f61-01ffef18da5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:07.230884 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca2e9d-a129-4355-9f61-01ffef18da5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:07.230884 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca2e9d-a129-4355-9f61-01ffef18da5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:07.230884 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca2e9d-a129-4355-9f61-01ffef18da5f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:07.233584 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 816/1628] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:07 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:07.234544 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ca90a-45f4-4ded-944d-22afb6311c39 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:07.245966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 813/1629] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:07.247127 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca2e9d-a129-4355-9f61-01ffef18da5f admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:07.253755 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ca90a-45f4-4ded-944d-22afb6311c39 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:07.264252 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7501822-fbbf-4320-9a7a-5d8d4e118550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:07.265161 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7501822-fbbf-4320-9a7a-5d8d4e118550 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:07.267810 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca2e9d-a129-4355-9f61-01ffef18da5f admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:07.270964 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7501822-fbbf-4320-9a7a-5d8d4e118550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:07.271377 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7501822-fbbf-4320-9a7a-5d8d4e118550 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:07.274421 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b27bb7-7bc4-4079-81b9-c23127e7bb0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:07.274941 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7501822-fbbf-4320-9a7a-5d8d4e118550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:07.275559 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b27bb7-7bc4-4079-81b9-c23127e7bb0f service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:07.275919 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b27bb7-7bc4-4079-81b9-c23127e7bb0f service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:07.276329 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b27bb7-7bc4-4079-81b9-c23127e7bb0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:07.278314 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7501822-fbbf-4320-9a7a-5d8d4e118550 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:07.322737 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 817/1630] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:07.322737 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7501822-fbbf-4320-9a7a-5d8d4e118550 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:07.333250 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7501822-fbbf-4320-9a7a-5d8d4e118550 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:07.339985 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129f009-cc85-43cc-a6aa-d8c7fadb6cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:07.340682 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129f009-cc85-43cc-a6aa-d8c7fadb6cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:07.341214 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129f009-cc85-43cc-a6aa-d8c7fadb6cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:07.341967 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129f009-cc85-43cc-a6aa-d8c7fadb6cb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:07.342890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c129f009-cc85-43cc-a6aa-d8c7fadb6cb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:07.344892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c129f009-cc85-43cc-a6aa-d8c7fadb6cb3 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:07.358721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 818/1631] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:07.359830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129f009-cc85-43cc-a6aa-d8c7fadb6cb3 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:07.369495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b27bb7-7bc4-4079-81b9-c23127e7bb0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:07.371469 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b27bb7-7bc4-4079-81b9-c23127e7bb0f service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:07.371626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129f009-cc85-43cc-a6aa-d8c7fadb6cb3 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:07.377493 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33028218-b72d-4aa6-a945-b4ce9260a688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:07.378619 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33028218-b72d-4aa6-a945-b4ce9260a688 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:07.379892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33028218-b72d-4aa6-a945-b4ce9260a688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:07.380495 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33028218-b72d-4aa6-a945-b4ce9260a688 admin admin] PATH_INFO: `/v3/projects/4ec6013ce82b48da8a79d818ab09bd0b/users/dac74f4ed1434280a87263c79487e172/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:07.383324 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 814/1632] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:07.384359 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b27bb7-7bc4-4079-81b9-c23127e7bb0f service nova] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:07.395602 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b27bb7-7bc4-4079-81b9-c23127e7bb0f service nova] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:07.399896 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33028218-b72d-4aa6-a945-b4ce9260a688 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ec6013ce82b48da8a79d818ab09bd0b, user_id=dac74f4ed1434280a87263c79487e172, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:07.403284 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33028218-b72d-4aa6-a945-b4ce9260a688 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:07.407305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff676f5-8de7-4b38-a842-0b57c7ad7baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:07.408375 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff676f5-8de7-4b38-a842-0b57c7ad7baa admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:07.408661 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff676f5-8de7-4b38-a842-0b57c7ad7baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:07.408932 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff676f5-8de7-4b38-a842-0b57c7ad7baa admin admin] PATH_INFO: `/v3/projects/273b7489aee041468a745ea43761cb02/users/05a72bbc289d40d19d1ac2b687536e99/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:07.414011 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:07.429087 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff676f5-8de7-4b38-a842-0b57c7ad7baa admin admin] RBAC: Authorizing `identity:create_grant(project_id=273b7489aee041468a745ea43761cb02, user_id=05a72bbc289d40d19d1ac2b687536e99, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:07.432473 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff676f5-8de7-4b38-a842-0b57c7ad7baa admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:07.533457 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 815/1633] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:07 2022] PUT /identity/v3/projects/273b7489aee041468a745ea43761cb02/users/05a72bbc289d40d19d1ac2b687536e99/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:07.534285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff676f5-8de7-4b38-a842-0b57c7ad7baa admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:07.536419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 819/1634] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:07 2022] PUT /identity/v3/projects/4ec6013ce82b48da8a79d818ab09bd0b/users/dac74f4ed1434280a87263c79487e172/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:07.540224 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33028218-b72d-4aa6-a945-b4ce9260a688 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:07.639471 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:07.664753 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33028218-b72d-4aa6-a945-b4ce9260a688 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:07.672525 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff676f5-8de7-4b38-a842-0b57c7ad7baa admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:07.676553 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e7ee4-513f-4775-bbad-87aa99e56e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:07.677332 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e7ee4-513f-4775-bbad-87aa99e56e7c admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:07.677517 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7013a477-9f08-4620-97b4-42fb70fc8d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:07.678170 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e7ee4-513f-4775-bbad-87aa99e56e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:07.678407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e7ee4-513f-4775-bbad-87aa99e56e7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:07.679409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40e7ee4-513f-4775-bbad-87aa99e56e7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:07.679586 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013a477-9f08-4620-97b4-42fb70fc8d97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:07.679982 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013a477-9f08-4620-97b4-42fb70fc8d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:07.680462 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7013a477-9f08-4620-97b4-42fb70fc8d97 admin admin] PATH_INFO: `/v3/projects/6d476e8cf4584c879fd0b8ad2e05df47/users/590117a74b744ede8574fc98a10aacce/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:07.681020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40e7ee4-513f-4775-bbad-87aa99e56e7c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:07.703179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 820/1635] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:35:07.705548 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e7ee4-513f-4775-bbad-87aa99e56e7c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:07.719611 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7013a477-9f08-4620-97b4-42fb70fc8d97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d476e8cf4584c879fd0b8ad2e05df47, user_id=590117a74b744ede8574fc98a10aacce, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:07.724246 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7013a477-9f08-4620-97b4-42fb70fc8d97 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:07.728233 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e7ee4-513f-4775-bbad-87aa99e56e7c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:07.749127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c341ea-b7af-4fa9-8047-af99538f4dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:07.750074 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c341ea-b7af-4fa9-8047-af99538f4dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:07.751460 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c341ea-b7af-4fa9-8047-af99538f4dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:07.751938 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c341ea-b7af-4fa9-8047-af99538f4dc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:07.752832 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c341ea-b7af-4fa9-8047-af99538f4dc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:07.757008 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c341ea-b7af-4fa9-8047-af99538f4dc9 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:07.781626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 821/1636] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:07.783778 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c341ea-b7af-4fa9-8047-af99538f4dc9 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:07.795695 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c341ea-b7af-4fa9-8047-af99538f4dc9 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:07.807678 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa65e0-c08d-4859-9a29-7399c71b8f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:07.808922 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa65e0-c08d-4859-9a29-7399c71b8f4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:07.808922 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa65e0-c08d-4859-9a29-7399c71b8f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:07.809115 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa65e0-c08d-4859-9a29-7399c71b8f4f admin admin] PATH_INFO: `/v3/projects/4ec6013ce82b48da8a79d818ab09bd0b/users/dac74f4ed1434280a87263c79487e172/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:07.816801 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa65e0-c08d-4859-9a29-7399c71b8f4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ec6013ce82b48da8a79d818ab09bd0b, user_id=dac74f4ed1434280a87263c79487e172, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:07.822129 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa65e0-c08d-4859-9a29-7399c71b8f4f admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:07.838204 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 822/1637] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:07 2022] PUT /identity/v3/projects/4ec6013ce82b48da8a79d818ab09bd0b/users/dac74f4ed1434280a87263c79487e172/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:07.840806 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa65e0-c08d-4859-9a29-7399c71b8f4f admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:07.868815 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 816/1638] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:07 2022] PUT /identity/v3/projects/6d476e8cf4584c879fd0b8ad2e05df47/users/590117a74b744ede8574fc98a10aacce/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:07.876195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d39cb8-1399-471e-8a80-46eeb7dc2a28 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:07.876195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d39cb8-1399-471e-8a80-46eeb7dc2a28 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:07.879053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d39cb8-1399-471e-8a80-46eeb7dc2a28 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:07.884276 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 817/1639] 10.222.0.24 () {54 vars in 912 bytes} [Fri May 27 03:35:07 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:35:07.886278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bf96d-0d81-4b0e-8f73-893065aaaeb4 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:07.886498 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bf96d-0d81-4b0e-8f73-893065aaaeb4 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:07.886700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bf96d-0d81-4b0e-8f73-893065aaaeb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:07.928568 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e45bf96d-0d81-4b0e-8f73-893065aaaeb4 None None] MFA Rules not processed for user `dac74f4ed1434280a87263c79487e172`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:07.930591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e45bf96d-0d81-4b0e-8f73-893065aaaeb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:07.955663 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa65e0-c08d-4859-9a29-7399c71b8f4f admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:08.050034 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e32b59-1dea-4bb9-9fcc-f29b8c7b92ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:08.051260 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e32b59-1dea-4bb9-9fcc-f29b8c7b92ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:08.051549 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e32b59-1dea-4bb9-9fcc-f29b8c7b92ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:08.051818 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e32b59-1dea-4bb9-9fcc-f29b8c7b92ed admin admin] PATH_INFO: `/v3/projects/273b7489aee041468a745ea43761cb02/users/05a72bbc289d40d19d1ac2b687536e99/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:08.054734 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e32b59-1dea-4bb9-9fcc-f29b8c7b92ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=273b7489aee041468a745ea43761cb02, user_id=05a72bbc289d40d19d1ac2b687536e99, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:08.057496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e32b59-1dea-4bb9-9fcc-f29b8c7b92ed admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:08.086461 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 818/1640] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:07 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:08.089290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45bf96d-0d81-4b0e-8f73-893065aaaeb4 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:08.095719 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 823/1641] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:07 2022] PUT /identity/v3/projects/273b7489aee041468a745ea43761cb02/users/05a72bbc289d40d19d1ac2b687536e99/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:08.098191 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a873d1-b7db-41e0-9c3f-750237d4c129 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:08.098662 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a873d1-b7db-41e0-9c3f-750237d4c129 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:08.098993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a873d1-b7db-41e0-9c3f-750237d4c129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:08.110467 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45bf96d-0d81-4b0e-8f73-893065aaaeb4 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:08.118733 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:08.125573 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-29a873d1-b7db-41e0-9c3f-750237d4c129 None None] MFA Rules not processed for user `22a9592143014f3287cd75c60ca17237`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:08.127410 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-29a873d1-b7db-41e0-9c3f-750237d4c129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:08.247682 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4192bde-8dc4-4229-b4fd-dda8a10a8330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:08.248527 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4192bde-8dc4-4229-b4fd-dda8a10a8330 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:08.248845 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4192bde-8dc4-4229-b4fd-dda8a10a8330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:08.249120 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4192bde-8dc4-4229-b4fd-dda8a10a8330 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:08.250276 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4192bde-8dc4-4229-b4fd-dda8a10a8330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:08.251951 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4192bde-8dc4-4229-b4fd-dda8a10a8330 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:08.266357 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 819/1642] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:08.266357 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71975375-f197-4b15-a857-ccf0b9f5a2ca None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:08.266357 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71975375-f197-4b15-a857-ccf0b9f5a2ca None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:08.266357 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71975375-f197-4b15-a857-ccf0b9f5a2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:08.268429 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 824/1643] 10.222.0.24 () {58 vars in 1009 bytes} [Fri May 27 03:35:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:08.269367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a873d1-b7db-41e0-9c3f-750237d4c129 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:08.302008 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-71975375-f197-4b15-a857-ccf0b9f5a2ca None None] MFA Rules not processed for user `05a72bbc289d40d19d1ac2b687536e99`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:08.304147 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-71975375-f197-4b15-a857-ccf0b9f5a2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:08.335784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a873d1-b7db-41e0-9c3f-750237d4c129 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:08.339707 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb0481a-7d71-4894-ab60-d9172d18bb9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:08.340769 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0481a-7d71-4894-ab60-d9172d18bb9f service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:08.341191 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0481a-7d71-4894-ab60-d9172d18bb9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:08.341468 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb0481a-7d71-4894-ab60-d9172d18bb9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:08.429942 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 820/1644] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:08 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:08.429942 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71975375-f197-4b15-a857-ccf0b9f5a2ca None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:08.439287 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71975375-f197-4b15-a857-ccf0b9f5a2ca None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:08.441761 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb0481a-7d71-4894-ab60-d9172d18bb9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:08.443367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb0481a-7d71-4894-ab60-d9172d18bb9f service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:08.446407 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf553a-081c-421e-ba73-01e1146ffc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:08.447282 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf553a-081c-421e-ba73-01e1146ffc76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:08.447666 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf553a-081c-421e-ba73-01e1146ffc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:08.449177 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf553a-081c-421e-ba73-01e1146ffc76 admin admin] PATH_INFO: `/v3/projects/6d476e8cf4584c879fd0b8ad2e05df47/users/590117a74b744ede8574fc98a10aacce/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:08.453720 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf553a-081c-421e-ba73-01e1146ffc76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d476e8cf4584c879fd0b8ad2e05df47, user_id=590117a74b744ede8574fc98a10aacce, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:08.458644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf553a-081c-421e-ba73-01e1146ffc76 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:08.470424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 825/1645] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:08 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:08.481779 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 821/1646] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:08 2022] PUT /identity/v3/projects/6d476e8cf4584c879fd0b8ad2e05df47/users/590117a74b744ede8574fc98a10aacce/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:08.498105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d206e6-d0f4-46c0-ac93-138c16df7787 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:08.498397 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d206e6-d0f4-46c0-ac93-138c16df7787 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:08.499220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d206e6-d0f4-46c0-ac93-138c16df7787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:08.556133 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-f7d206e6-d0f4-46c0-ac93-138c16df7787 None None] MFA Rules not processed for user `590117a74b744ede8574fc98a10aacce`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:08.558905 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f7d206e6-d0f4-46c0-ac93-138c16df7787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:08.567771 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf553a-081c-421e-ba73-01e1146ffc76 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:08.600957 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:08.667907 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf553a-081c-421e-ba73-01e1146ffc76 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:08.684643 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388367e-650d-49cf-898d-42757c9228a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:08.685792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388367e-650d-49cf-898d-42757c9228a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:08.686299 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388367e-650d-49cf-898d-42757c9228a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:08.687058 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388367e-650d-49cf-898d-42757c9228a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:08.705193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 826/1647] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:08 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:08.705193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d206e6-d0f4-46c0-ac93-138c16df7787 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:08.718025 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d206e6-d0f4-46c0-ac93-138c16df7787 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:08.722302 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f61178-42c4-4186-9a1d-908f3401ae0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:08.723197 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f61178-42c4-4186-9a1d-908f3401ae0c service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:08.723960 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f61178-42c4-4186-9a1d-908f3401ae0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:08.723960 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f61178-42c4-4186-9a1d-908f3401ae0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:08.821105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f61178-42c4-4186-9a1d-908f3401ae0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:08.823222 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f61178-42c4-4186-9a1d-908f3401ae0c service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:08.827968 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388367e-650d-49cf-898d-42757c9228a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:08.829934 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388367e-650d-49cf-898d-42757c9228a0 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:08.837250 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 827/1648] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:08 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:08.844172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 822/1649] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:09.092605 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f61178-42c4-4186-9a1d-908f3401ae0c service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:09.092605 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f61178-42c4-4186-9a1d-908f3401ae0c service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:09.092605 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737a9b8-5dab-40d1-9dd1-a4336bc4ad60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:09.092605 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737a9b8-5dab-40d1-9dd1-a4336bc4ad60 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:09.098162 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737a9b8-5dab-40d1-9dd1-a4336bc4ad60 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:09.098162 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737a9b8-5dab-40d1-9dd1-a4336bc4ad60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:09.098162 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737a9b8-5dab-40d1-9dd1-a4336bc4ad60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:09.098162 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737a9b8-5dab-40d1-9dd1-a4336bc4ad60 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:09.105067 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16be5b-3fe9-4e5b-8de5-3e15959e43e9 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:09.105067 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16be5b-3fe9-4e5b-8de5-3e15959e43e9 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:09.105067 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d16be5b-3fe9-4e5b-8de5-3e15959e43e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:09.111260 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 828/1650] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:09 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:09.125985 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:09.166434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-3d16be5b-3fe9-4e5b-8de5-3e15959e43e9 None None] MFA Rules not processed for user `8584a3f99b0144979f69536f7cc86851`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:09.166995 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3d16be5b-3fe9-4e5b-8de5-3e15959e43e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:09.323209 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 823/1651] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:35:09.331703 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066533b-644e-4e63-ad25-dce3933f3e2a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:09.332024 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066533b-644e-4e63-ad25-dce3933f3e2a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:09.332262 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066533b-644e-4e63-ad25-dce3933f3e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:09.360470 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-3066533b-644e-4e63-ad25-dce3933f3e2a None None] MFA Rules not processed for user `8584a3f99b0144979f69536f7cc86851`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:09.361698 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3066533b-644e-4e63-ad25-dce3933f3e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:09.382457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 829/1652] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:09.388901 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16be5b-3fe9-4e5b-8de5-3e15959e43e9 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:09.411233 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d16be5b-3fe9-4e5b-8de5-3e15959e43e9 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:09.433602 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b7c32-b0e6-4317-b1dd-ef41597379c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:09.438945 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b7c32-b0e6-4317-b1dd-ef41597379c5 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:09.438945 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b7c32-b0e6-4317-b1dd-ef41597379c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:09.438945 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b7c32-b0e6-4317-b1dd-ef41597379c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:09.468499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598b7c32-b0e6-4317-b1dd-ef41597379c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:09.481752 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598b7c32-b0e6-4317-b1dd-ef41597379c5 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:09.490984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066533b-644e-4e63-ad25-dce3933f3e2a None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:09.511842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 824/1653] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:09.613473 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b7c32-b0e6-4317-b1dd-ef41597379c5 service nova] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:09.725362 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066533b-644e-4e63-ad25-dce3933f3e2a None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:09.736520 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7fcf4c-1426-4eef-8496-80577834f526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:09.737627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fcf4c-1426-4eef-8496-80577834f526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:09.739102 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fcf4c-1426-4eef-8496-80577834f526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:09.740374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fcf4c-1426-4eef-8496-80577834f526 admin admin] PATH_INFO: `/v3/users/78120e5efd9a429ebf508a26d2b7bc6b` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:09.744061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7fcf4c-1426-4eef-8496-80577834f526 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78120e5efd9a429ebf508a26d2b7bc6b)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:09.746942 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: WARNING py.warnings [None req-de7fcf4c-1426-4eef-8496-80577834f526 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:35:09.746942 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: warnings.warn(msg) May 27 03:35:09.746942 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: May 27 03:35:09.749639 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7fcf4c-1426-4eef-8496-80577834f526 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:09.770798 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:09.809795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b7c32-b0e6-4317-b1dd-ef41597379c5 service nova] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:09.815496 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252efd58-c139-42e3-bccb-f9f96693b09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:09.816488 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252efd58-c139-42e3-bccb-f9f96693b09c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:09.816873 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252efd58-c139-42e3-bccb-f9f96693b09c admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:09.817267 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252efd58-c139-42e3-bccb-f9f96693b09c admin admin] PATH_INFO: `/v3/users/8e5f5e4449f9455e82397a4137e3cf84` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:09.822062 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252efd58-c139-42e3-bccb-f9f96693b09c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e5f5e4449f9455e82397a4137e3cf84)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:09.824609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252efd58-c139-42e3-bccb-f9f96693b09c admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:10.041513 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-252efd58-c139-42e3-bccb-f9f96693b09c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e5f5e4449f9455e82397a4137e3cf84'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:10.083661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-de7fcf4c-1426-4eef-8496-80577834f526 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78120e5efd9a429ebf508a26d2b7bc6b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:10.092664 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:10.121092 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-252efd58-c139-42e3-bccb-f9f96693b09c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e5f5e4449f9455e82397a4137e3cf84'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:10.121739 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-252efd58-c139-42e3-bccb-f9f96693b09c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e5f5e4449f9455e82397a4137e3cf84'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:10.139898 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-252efd58-c139-42e3-bccb-f9f96693b09c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e5f5e4449f9455e82397a4137e3cf84'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:10.140694 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-252efd58-c139-42e3-bccb-f9f96693b09c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e5f5e4449f9455e82397a4137e3cf84'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:10.178397 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 825/1654] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:35:09 2022] DELETE /identity/v3/users/8e5f5e4449f9455e82397a4137e3cf84 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:35:10.179302 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252efd58-c139-42e3-bccb-f9f96693b09c admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:10.180587 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-252efd58-c139-42e3-bccb-f9f96693b09c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:10.209376 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-de7fcf4c-1426-4eef-8496-80577834f526 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78120e5efd9a429ebf508a26d2b7bc6b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:10.292287 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-de7fcf4c-1426-4eef-8496-80577834f526 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78120e5efd9a429ebf508a26d2b7bc6b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:10.296468 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-de7fcf4c-1426-4eef-8496-80577834f526 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78120e5efd9a429ebf508a26d2b7bc6b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:10.314594 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252efd58-c139-42e3-bccb-f9f96693b09c admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:10.320559 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7cdf7-f3b5-4828-ad99-bdc7dc5772c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:10.321342 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7cdf7-f3b5-4828-ad99-bdc7dc5772c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:10.321518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7cdf7-f3b5-4828-ad99-bdc7dc5772c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:10.321796 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7cdf7-f3b5-4828-ad99-bdc7dc5772c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:10.328435 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-9ee7cdf7-f3b5-4828-ad99-bdc7dc5772c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:10.333659 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-de7fcf4c-1426-4eef-8496-80577834f526 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78120e5efd9a429ebf508a26d2b7bc6b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:10.363021 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 830/1655] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:35:09 2022] DELETE /identity/v3/users/78120e5efd9a429ebf508a26d2b7bc6b => generated 0 bytes in 870 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:35:10.367338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f40959-ecdb-4d6f-8c07-c02663531405 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:10.367510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f40959-ecdb-4d6f-8c07-c02663531405 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:10.367726 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f40959-ecdb-4d6f-8c07-c02663531405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:10.409703 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-f8f40959-ecdb-4d6f-8c07-c02663531405 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:10.418293 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f8f40959-ecdb-4d6f-8c07-c02663531405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:10.470730 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee7cdf7-f3b5-4828-ad99-bdc7dc5772c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:10.473056 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee7cdf7-f3b5-4828-ad99-bdc7dc5772c4 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:10.493502 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 826/1656] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:10 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:10.495901 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffb0e4-42f5-4582-91c0-999946fe798b None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:10.496191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffb0e4-42f5-4582-91c0-999946fe798b None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:10.499130 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffb0e4-42f5-4582-91c0-999946fe798b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:10.547175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-50ffb0e4-42f5-4582-91c0-999946fe798b None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:10.548402 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-50ffb0e4-42f5-4582-91c0-999946fe798b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:10.673620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 831/1657] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:10.679359 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac96b9-2ca6-471a-b9a3-53d7f2f69a82 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:10.679896 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac96b9-2ca6-471a-b9a3-53d7f2f69a82 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:10.680413 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac96b9-2ca6-471a-b9a3-53d7f2f69a82 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:10.691722 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 832/1658] 10.222.0.24 () {54 vars in 912 bytes} [Fri May 27 03:35:10 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:35:10.692945 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac96b9-2ca6-471a-b9a3-53d7f2f69a82 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:10.699001 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5fac96b9-2ca6-471a-b9a3-53d7f2f69a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:10.709493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 827/1659] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:10.713832 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85008cb6-b8bf-48c0-b28f-aedc508ebdd9 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:10.713832 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85008cb6-b8bf-48c0-b28f-aedc508ebdd9 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:10.713832 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85008cb6-b8bf-48c0-b28f-aedc508ebdd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:10.803940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-85008cb6-b8bf-48c0-b28f-aedc508ebdd9 None None] MFA Rules not processed for user `1af0160cd3da4a0bb391b81f083715d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:10.805179 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-85008cb6-b8bf-48c0-b28f-aedc508ebdd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:10.808551 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac96b9-2ca6-471a-b9a3-53d7f2f69a82 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:10.815296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920c552-f9db-457c-9b8e-85227a5fb527 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:10.822811 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920c552-f9db-457c-9b8e-85227a5fb527 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:10.822811 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920c552-f9db-457c-9b8e-85227a5fb527 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:10.822811 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c920c552-f9db-457c-9b8e-85227a5fb527 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:10.826292 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 828/1660] 10.222.0.24 () {58 vars in 1009 bytes} [Fri May 27 03:35:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:10.827022 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85008cb6-b8bf-48c0-b28f-aedc508ebdd9 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:10.839881 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-85008cb6-b8bf-48c0-b28f-aedc508ebdd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:10.851280 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85008cb6-b8bf-48c0-b28f-aedc508ebdd9 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:10.861519 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5fec33-d54d-468b-85b1-81cde6d9cb45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:10.862882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5fec33-d54d-468b-85b1-81cde6d9cb45 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:10.863541 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5fec33-d54d-468b-85b1-81cde6d9cb45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:10.863989 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5fec33-d54d-468b-85b1-81cde6d9cb45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:10.871022 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c920c552-f9db-457c-9b8e-85227a5fb527 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:10.873227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c920c552-f9db-457c-9b8e-85227a5fb527 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:10.896875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 833/1661] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:10.898029 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920c552-f9db-457c-9b8e-85227a5fb527 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:10.908549 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5fec33-d54d-468b-85b1-81cde6d9cb45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:10.911198 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5fec33-d54d-468b-85b1-81cde6d9cb45 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:10.913109 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c920c552-f9db-457c-9b8e-85227a5fb527 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:10.917807 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd58809-15f2-4131-9812-40cbd1b13734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:10.922696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:10.935236 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 829/1662] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:10.937442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd58809-15f2-4131-9812-40cbd1b13734 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:10.937704 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd58809-15f2-4131-9812-40cbd1b13734 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:10.938883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd58809-15f2-4131-9812-40cbd1b13734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:10.974045 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd58809-15f2-4131-9812-40cbd1b13734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:10.975765 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd58809-15f2-4131-9812-40cbd1b13734 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:10.997086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 834/1663] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:11.414091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5fec33-d54d-468b-85b1-81cde6d9cb45 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:11.417094 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6d5fec33-d54d-468b-85b1-81cde6d9cb45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:11.434092 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5fec33-d54d-468b-85b1-81cde6d9cb45 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:11.440722 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3fd8c4-cca7-4a0c-b610-feb3c42bfa35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:11.441916 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3fd8c4-cca7-4a0c-b610-feb3c42bfa35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:11.442394 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3fd8c4-cca7-4a0c-b610-feb3c42bfa35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:11.442804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3fd8c4-cca7-4a0c-b610-feb3c42bfa35 admin admin] PATH_INFO: `/v3/users/fb44774f48984906b23c799d139fdd27` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:11.444990 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3fd8c4-cca7-4a0c-b610-feb3c42bfa35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb44774f48984906b23c799d139fdd27)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:11.447353 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3fd8c4-cca7-4a0c-b610-feb3c42bfa35 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:11.739971 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-4f3fd8c4-cca7-4a0c-b610-feb3c42bfa35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb44774f48984906b23c799d139fdd27'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:11.841271 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-4f3fd8c4-cca7-4a0c-b610-feb3c42bfa35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb44774f48984906b23c799d139fdd27'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:11.841971 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-4f3fd8c4-cca7-4a0c-b610-feb3c42bfa35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb44774f48984906b23c799d139fdd27'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:11.853036 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-4f3fd8c4-cca7-4a0c-b610-feb3c42bfa35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb44774f48984906b23c799d139fdd27'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:11.854570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-4f3fd8c4-cca7-4a0c-b610-feb3c42bfa35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb44774f48984906b23c799d139fdd27'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:11.891201 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 830/1664] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:35:11 2022] DELETE /identity/v3/users/fb44774f48984906b23c799d139fdd27 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:11.895956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd40843-b9c2-4a52-a034-3bd83a000c9d None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:11.897141 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd40843-b9c2-4a52-a034-3bd83a000c9d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:11.897219 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd40843-b9c2-4a52-a034-3bd83a000c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:11.925970 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-edd40843-b9c2-4a52-a034-3bd83a000c9d None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:11.927616 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-edd40843-b9c2-4a52-a034-3bd83a000c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:12.021248 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:12.043162 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 835/1665] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:12.052266 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3fd8c4-cca7-4a0c-b610-feb3c42bfa35 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:12.054369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-4f3fd8c4-cca7-4a0c-b610-feb3c42bfa35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:12.072737 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd40843-b9c2-4a52-a034-3bd83a000c9d None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:12.076848 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-edd40843-b9c2-4a52-a034-3bd83a000c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:12.103966 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd40843-b9c2-4a52-a034-3bd83a000c9d None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:12.116886 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca886e-2647-4e50-912f-129f36ca90c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:12.118262 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca886e-2647-4e50-912f-129f36ca90c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:12.118547 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca886e-2647-4e50-912f-129f36ca90c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:12.118983 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca886e-2647-4e50-912f-129f36ca90c4 admin admin] PATH_INFO: `/v3/projects/2adb235324be4907ac4e0be5e0e901dd` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:12.124511 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca886e-2647-4e50-912f-129f36ca90c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2adb235324be4907ac4e0be5e0e901dd)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:12.131968 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca886e-2647-4e50-912f-129f36ca90c4 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:12.248509 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3fd8c4-cca7-4a0c-b610-feb3c42bfa35 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:12.264791 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80da8f-3852-4483-9f71-4fde51529786 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:12.265621 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80da8f-3852-4483-9f71-4fde51529786 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:12.265908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80da8f-3852-4483-9f71-4fde51529786 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:12.266180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d80da8f-3852-4483-9f71-4fde51529786 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:12.304254 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80da8f-3852-4483-9f71-4fde51529786 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:12.305990 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d80da8f-3852-4483-9f71-4fde51529786 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:12.341525 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 831/1666] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:12.348347 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80da8f-3852-4483-9f71-4fde51529786 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:12.352091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-5d80da8f-3852-4483-9f71-4fde51529786 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:12.382363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7eca886e-2647-4e50-912f-129f36ca90c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2adb235324be4907ac4e0be5e0e901dd', 'request_id': 'req-7eca886e-2647-4e50-912f-129f36ca90c4'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:12.391102 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d80da8f-3852-4483-9f71-4fde51529786 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:12.535328 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7eca886e-2647-4e50-912f-129f36ca90c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2adb235324be4907ac4e0be5e0e901dd', 'request_id': 'req-7eca886e-2647-4e50-912f-129f36ca90c4'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:12.543810 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7eca886e-2647-4e50-912f-129f36ca90c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2adb235324be4907ac4e0be5e0e901dd', 'request_id': 'req-7eca886e-2647-4e50-912f-129f36ca90c4'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:12.571254 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7eca886e-2647-4e50-912f-129f36ca90c4 admin admin] The token cache is being invalidate because project 2adb235324be4907ac4e0be5e0e901dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:35:12.571550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7eca886e-2647-4e50-912f-129f36ca90c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:12.572260 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7eca886e-2647-4e50-912f-129f36ca90c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:12.579892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 836/1667] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:35:12 2022] DELETE /identity/v3/projects/2adb235324be4907ac4e0be5e0e901dd => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:12.581239 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e4666-0f91-4454-adc7-09d3fa107dc3 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:12.581589 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e4666-0f91-4454-adc7-09d3fa107dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:12.581875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e4666-0f91-4454-adc7-09d3fa107dc3 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:12.588559 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 837/1668] 10.222.0.24 () {54 vars in 960 bytes} [Fri May 27 03:35:12 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:35:12.590180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e4666-0f91-4454-adc7-09d3fa107dc3 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:12.595796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-747e4666-0f91-4454-adc7-09d3fa107dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:12.600851 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f95112-6233-44ad-a866-10063a615f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:12.605077 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f95112-6233-44ad-a866-10063a615f88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:12.605641 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f95112-6233-44ad-a866-10063a615f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:12.605933 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f95112-6233-44ad-a866-10063a615f88 admin admin] PATH_INFO: `/v3/users/1d6495f63468440991b39a43a11c856b` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:12.609096 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f95112-6233-44ad-a866-10063a615f88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d6495f63468440991b39a43a11c856b)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:12.619211 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f95112-6233-44ad-a866-10063a615f88 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:12.648763 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e4666-0f91-4454-adc7-09d3fa107dc3 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:12.666282 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a519b085-2686-4ec1-8cd1-3a6bc8b6e063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:12.667397 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519b085-2686-4ec1-8cd1-3a6bc8b6e063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:12.667724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519b085-2686-4ec1-8cd1-3a6bc8b6e063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:12.667999 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519b085-2686-4ec1-8cd1-3a6bc8b6e063 admin admin] PATH_INFO: `/v3/projects/c1d79b4ed73547e8ac943686bc7661df` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:12.669818 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a519b085-2686-4ec1-8cd1-3a6bc8b6e063 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1d79b4ed73547e8ac943686bc7661df)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:12.671978 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a519b085-2686-4ec1-8cd1-3a6bc8b6e063 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:12.733259 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:12.873929 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-81f95112-6233-44ad-a866-10063a615f88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d6495f63468440991b39a43a11c856b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:12.885266 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a519b085-2686-4ec1-8cd1-3a6bc8b6e063 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1d79b4ed73547e8ac943686bc7661df', 'request_id': 'req-a519b085-2686-4ec1-8cd1-3a6bc8b6e063'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:12.990475 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-81f95112-6233-44ad-a866-10063a615f88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d6495f63468440991b39a43a11c856b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:12.991484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-81f95112-6233-44ad-a866-10063a615f88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d6495f63468440991b39a43a11c856b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:13.002810 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-81f95112-6233-44ad-a866-10063a615f88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d6495f63468440991b39a43a11c856b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:13.003968 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-81f95112-6233-44ad-a866-10063a615f88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d6495f63468440991b39a43a11c856b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:13.029526 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 832/1669] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:35:12 2022] DELETE /identity/v3/users/1d6495f63468440991b39a43a11c856b => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:13.030683 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f95112-6233-44ad-a866-10063a615f88 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:13.032371 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-81f95112-6233-44ad-a866-10063a615f88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:13.087449 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a519b085-2686-4ec1-8cd1-3a6bc8b6e063 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1d79b4ed73547e8ac943686bc7661df', 'request_id': 'req-a519b085-2686-4ec1-8cd1-3a6bc8b6e063'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:13.096891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a519b085-2686-4ec1-8cd1-3a6bc8b6e063 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1d79b4ed73547e8ac943686bc7661df', 'request_id': 'req-a519b085-2686-4ec1-8cd1-3a6bc8b6e063'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:13.114595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f95112-6233-44ad-a866-10063a615f88 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:13.121126 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6f962-4c13-4866-8b89-493e7af12980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:13.122238 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6f962-4c13-4866-8b89-493e7af12980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:13.122683 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6f962-4c13-4866-8b89-493e7af12980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:13.126089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6f962-4c13-4866-8b89-493e7af12980 admin admin] PATH_INFO: `/v3/users/05a72bbc289d40d19d1ac2b687536e99` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:13.126089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6f962-4c13-4866-8b89-493e7af12980 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05a72bbc289d40d19d1ac2b687536e99)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:13.127317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6f962-4c13-4866-8b89-493e7af12980 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:13.141777 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a519b085-2686-4ec1-8cd1-3a6bc8b6e063 admin admin] The token cache is being invalidate because project c1d79b4ed73547e8ac943686bc7661df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:35:13.142387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a519b085-2686-4ec1-8cd1-3a6bc8b6e063 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:13.144135 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a519b085-2686-4ec1-8cd1-3a6bc8b6e063 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:13.151845 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 838/1670] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:35:12 2022] DELETE /identity/v3/projects/c1d79b4ed73547e8ac943686bc7661df => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:13.158074 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3282c-038e-4c2d-9abc-de657b39f150 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:13.159601 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3282c-038e-4c2d-9abc-de657b39f150 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:13.159919 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3282c-038e-4c2d-9abc-de657b39f150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:13.224100 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-4ab3282c-038e-4c2d-9abc-de657b39f150 None None] MFA Rules not processed for user `95f2d0aceef4451a83ee89fac5130a0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:13.226046 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4ab3282c-038e-4c2d-9abc-de657b39f150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:13.345967 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:13.347901 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e7e6f962-4c13-4866-8b89-493e7af12980 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05a72bbc289d40d19d1ac2b687536e99'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:13.375644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 839/1671] 10.222.0.24 () {58 vars in 1057 bytes} [Fri May 27 03:35:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:13.377579 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cea333-58ae-4776-8dad-afa442edee5a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:13.377971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cea333-58ae-4776-8dad-afa442edee5a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:13.378874 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cea333-58ae-4776-8dad-afa442edee5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:13.410708 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-61cea333-58ae-4776-8dad-afa442edee5a None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:13.418380 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-61cea333-58ae-4776-8dad-afa442edee5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:13.437629 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e7e6f962-4c13-4866-8b89-493e7af12980 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05a72bbc289d40d19d1ac2b687536e99'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:13.438292 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e7e6f962-4c13-4866-8b89-493e7af12980 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05a72bbc289d40d19d1ac2b687536e99'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:13.443829 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 840/1672] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:13.445148 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac08f2-52ac-4c99-bfe0-4d4356180a17 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:13.445435 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac08f2-52ac-4c99-bfe0-4d4356180a17 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:13.446761 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac08f2-52ac-4c99-bfe0-4d4356180a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:13.455586 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e7e6f962-4c13-4866-8b89-493e7af12980 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05a72bbc289d40d19d1ac2b687536e99'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:13.456650 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e7e6f962-4c13-4866-8b89-493e7af12980 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05a72bbc289d40d19d1ac2b687536e99'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:13.496526 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 833/1673] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:35:13 2022] DELETE /identity/v3/users/05a72bbc289d40d19d1ac2b687536e99 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:13.497711 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6f962-4c13-4866-8b89-493e7af12980 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:13.504239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e7e6f962-4c13-4866-8b89-493e7af12980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:13.527876 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-21ac08f2-52ac-4c99-bfe0-4d4356180a17 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:13.530986 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-21ac08f2-52ac-4c99-bfe0-4d4356180a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:13.681589 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6f962-4c13-4866-8b89-493e7af12980 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:13.695046 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db2828-12b7-460b-8462-437f6990606f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f2d0aceef4451a83ee89fac5130a0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:13.695986 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db2828-12b7-460b-8462-437f6990606f service placement] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:13.696485 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db2828-12b7-460b-8462-437f6990606f service placement] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:13.696979 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db2828-12b7-460b-8462-437f6990606f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:13.700323 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-77db2828-12b7-460b-8462-437f6990606f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:13.731205 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77db2828-12b7-460b-8462-437f6990606f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:13.733385 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77db2828-12b7-460b-8462-437f6990606f service placement] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:13.740760 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 841/1674] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:13.748051 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac08f2-52ac-4c99-bfe0-4d4356180a17 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:13.749803 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-21ac08f2-52ac-4c99-bfe0-4d4356180a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:13.776174 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 834/1675] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:13.777486 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db2828-12b7-460b-8462-437f6990606f service placement] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:13.785999 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-77db2828-12b7-460b-8462-437f6990606f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:13.837812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db2828-12b7-460b-8462-437f6990606f service placement] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:13.873254 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77542a66-b7bd-4f38-8903-d0ec6b12833c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:13.874156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77542a66-b7bd-4f38-8903-d0ec6b12833c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:13.874768 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77542a66-b7bd-4f38-8903-d0ec6b12833c admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:13.874768 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77542a66-b7bd-4f38-8903-d0ec6b12833c admin admin] PATH_INFO: `/v3/projects/fe3e221d39d8476a9a79989e792590e9` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:13.883143 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77542a66-b7bd-4f38-8903-d0ec6b12833c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe3e221d39d8476a9a79989e792590e9)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:13.884206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: WARNING py.warnings [None req-77542a66-b7bd-4f38-8903-d0ec6b12833c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:35:13.884206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: warnings.warn(msg) May 27 03:35:13.884206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: May 27 03:35:13.885428 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77542a66-b7bd-4f38-8903-d0ec6b12833c admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:13.903752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac08f2-52ac-4c99-bfe0-4d4356180a17 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:13.923566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302a0be-9340-4a38-a89c-435156b9a884 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:13.924430 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302a0be-9340-4a38-a89c-435156b9a884 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:13.924784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302a0be-9340-4a38-a89c-435156b9a884 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:13.925133 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302a0be-9340-4a38-a89c-435156b9a884 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:13.928208 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5302a0be-9340-4a38-a89c-435156b9a884 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:13.965145 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5302a0be-9340-4a38-a89c-435156b9a884 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:13.970133 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5302a0be-9340-4a38-a89c-435156b9a884 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:13.997991 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 842/1676] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:14.001270 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc23f5-79c7-4a86-bc96-04f6656d4f71 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:14.002785 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc23f5-79c7-4a86-bc96-04f6656d4f71 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:14.004582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc23f5-79c7-4a86-bc96-04f6656d4f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:14.045990 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-58dc23f5-79c7-4a86-bc96-04f6656d4f71 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:14.047414 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-58dc23f5-79c7-4a86-bc96-04f6656d4f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:14.057678 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 843/1677] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:14.061609 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc23f5-79c7-4a86-bc96-04f6656d4f71 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:14.074041 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc23f5-79c7-4a86-bc96-04f6656d4f71 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:14.101881 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476eaa76-17c6-4f0b-a52b-de01c0d3a8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:14.102567 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476eaa76-17c6-4f0b-a52b-de01c0d3a8ea admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:14.103022 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476eaa76-17c6-4f0b-a52b-de01c0d3a8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:14.103303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476eaa76-17c6-4f0b-a52b-de01c0d3a8ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:14.104323 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476eaa76-17c6-4f0b-a52b-de01c0d3a8ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:35:14.104656 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476eaa76-17c6-4f0b-a52b-de01c0d3a8ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:14.108563 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476eaa76-17c6-4f0b-a52b-de01c0d3a8ea admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:14.134248 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-77542a66-b7bd-4f38-8903-d0ec6b12833c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe3e221d39d8476a9a79989e792590e9', 'request_id': 'req-77542a66-b7bd-4f38-8903-d0ec6b12833c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:14.146448 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 844/1678] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:35:14.151137 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476eaa76-17c6-4f0b-a52b-de01c0d3a8ea admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:14.168046 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476eaa76-17c6-4f0b-a52b-de01c0d3a8ea admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:14.223298 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-77542a66-b7bd-4f38-8903-d0ec6b12833c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe3e221d39d8476a9a79989e792590e9', 'request_id': 'req-77542a66-b7bd-4f38-8903-d0ec6b12833c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:14.261336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-77542a66-b7bd-4f38-8903-d0ec6b12833c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe3e221d39d8476a9a79989e792590e9', 'request_id': 'req-77542a66-b7bd-4f38-8903-d0ec6b12833c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:14.273044 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9cbfa2-54ec-486f-8e28-c9d312e7919a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:14.273937 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cbfa2-54ec-486f-8e28-c9d312e7919a admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:14.275293 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cbfa2-54ec-486f-8e28-c9d312e7919a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:14.276639 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9cbfa2-54ec-486f-8e28-c9d312e7919a admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:14.277549 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9cbfa2-54ec-486f-8e28-c9d312e7919a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:14.281837 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9cbfa2-54ec-486f-8e28-c9d312e7919a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:14.285136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-77542a66-b7bd-4f38-8903-d0ec6b12833c admin admin] The token cache is being invalidate because project fe3e221d39d8476a9a79989e792590e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:35:14.285467 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-77542a66-b7bd-4f38-8903-d0ec6b12833c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:14.289639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-77542a66-b7bd-4f38-8903-d0ec6b12833c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:14.300862 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 835/1679] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:35:13 2022] DELETE /identity/v3/projects/fe3e221d39d8476a9a79989e792590e9 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:14.300862 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77542a66-b7bd-4f38-8903-d0ec6b12833c admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:14.303814 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-77542a66-b7bd-4f38-8903-d0ec6b12833c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:14.356908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77542a66-b7bd-4f38-8903-d0ec6b12833c admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:14.365183 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b40391-75b3-4578-9a50-7447a957de45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:14.366218 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b40391-75b3-4578-9a50-7447a957de45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:14.366553 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b40391-75b3-4578-9a50-7447a957de45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:14.366990 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b40391-75b3-4578-9a50-7447a957de45 admin admin] PATH_INFO: `/v3/users/dac74f4ed1434280a87263c79487e172` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:14.369762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b40391-75b3-4578-9a50-7447a957de45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dac74f4ed1434280a87263c79487e172)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:14.372089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b40391-75b3-4578-9a50-7447a957de45 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:14.452504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 845/1680] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:14 2022] POST /identity/v3/projects => generated 403 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:14.460065 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9cbfa2-54ec-486f-8e28-c9d312e7919a admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:14.468012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1a9cbfa2-54ec-486f-8e28-c9d312e7919a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:14.633825 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9cbfa2-54ec-486f-8e28-c9d312e7919a admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:14.645705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b9349-5a06-4bae-9e00-fb0dfa013367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:14.647766 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b9349-5a06-4bae-9e00-fb0dfa013367 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:14.648188 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b9349-5a06-4bae-9e00-fb0dfa013367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:14.648777 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b9349-5a06-4bae-9e00-fb0dfa013367 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:14.649613 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b9349-5a06-4bae-9e00-fb0dfa013367 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:14.651705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b9349-5a06-4bae-9e00-fb0dfa013367 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:14.718072 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-66b40391-75b3-4578-9a50-7447a957de45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dac74f4ed1434280a87263c79487e172'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:14.781487 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:14.796216 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 846/1681] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:14 2022] POST /identity/v3/users => generated 302 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:14.797245 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b9349-5a06-4bae-9e00-fb0dfa013367 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:14.819057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b9349-5a06-4bae-9e00-fb0dfa013367 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:14.831374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929b40b-dea5-4847-9b82-320d92cd6c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:14.832152 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929b40b-dea5-4847-9b82-320d92cd6c52 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:14.832319 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-66b40391-75b3-4578-9a50-7447a957de45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dac74f4ed1434280a87263c79487e172'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:14.832486 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929b40b-dea5-4847-9b82-320d92cd6c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:14.832811 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929b40b-dea5-4847-9b82-320d92cd6c52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:14.833652 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-66b40391-75b3-4578-9a50-7447a957de45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dac74f4ed1434280a87263c79487e172'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:14.833781 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929b40b-dea5-4847-9b82-320d92cd6c52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:14.836977 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929b40b-dea5-4847-9b82-320d92cd6c52 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:14.847319 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-66b40391-75b3-4578-9a50-7447a957de45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dac74f4ed1434280a87263c79487e172'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:14.852514 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-66b40391-75b3-4578-9a50-7447a957de45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dac74f4ed1434280a87263c79487e172'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:14.861335 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 847/1682] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:14.863909 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929b40b-dea5-4847-9b82-320d92cd6c52 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:14.868073 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4929b40b-dea5-4847-9b82-320d92cd6c52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:14.915436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 836/1683] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:35:14 2022] DELETE /identity/v3/users/dac74f4ed1434280a87263c79487e172 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:14.917992 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929b40b-dea5-4847-9b82-320d92cd6c52 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:15.045019 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652ead3-56c3-4255-aeab-b7cdf2587139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:15.046936 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652ead3-56c3-4255-aeab-b7cdf2587139 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:15.047682 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652ead3-56c3-4255-aeab-b7cdf2587139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:15.048417 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652ead3-56c3-4255-aeab-b7cdf2587139 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:15.050132 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652ead3-56c3-4255-aeab-b7cdf2587139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:15.054265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652ead3-56c3-4255-aeab-b7cdf2587139 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:15.077311 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 848/1684] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:15.081157 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b40391-75b3-4578-9a50-7447a957de45 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:15.106413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b40391-75b3-4578-9a50-7447a957de45 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:15.136515 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e278a8-2a97-4f55-b053-8e4d7869d65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:15.137670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e278a8-2a97-4f55-b053-8e4d7869d65f admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:15.138373 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e278a8-2a97-4f55-b053-8e4d7869d65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:15.138797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e278a8-2a97-4f55-b053-8e4d7869d65f admin admin] PATH_INFO: `/v3/projects/46545b5ceca74e959623f6e9d053da68/users/402ec1b74c464319a5d4a5628f6f9e2a/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:15.178310 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e278a8-2a97-4f55-b053-8e4d7869d65f admin admin] RBAC: Authorizing `identity:create_grant(project_id=46545b5ceca74e959623f6e9d053da68, user_id=402ec1b74c464319a5d4a5628f6f9e2a, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:15.183528 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e278a8-2a97-4f55-b053-8e4d7869d65f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:15.304946 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 837/1685] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:15 2022] PUT /identity/v3/projects/46545b5ceca74e959623f6e9d053da68/users/402ec1b74c464319a5d4a5628f6f9e2a/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:15.315046 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09bc80-7d14-4d81-9965-93254c02d29e None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:15.315046 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09bc80-7d14-4d81-9965-93254c02d29e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:15.315046 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09bc80-7d14-4d81-9965-93254c02d29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:15.341668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e278a8-2a97-4f55-b053-8e4d7869d65f admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:15.343796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-4b09bc80-7d14-4d81-9965-93254c02d29e None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:15.344147 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f4e278a8-2a97-4f55-b053-8e4d7869d65f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:15.347658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4b09bc80-7d14-4d81-9965-93254c02d29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:15.477563 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e278a8-2a97-4f55-b053-8e4d7869d65f admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:15.484935 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9442c-f2cf-4536-accc-0bcf85dc68c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:15.486076 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9442c-f2cf-4536-accc-0bcf85dc68c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:15.487296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9442c-f2cf-4536-accc-0bcf85dc68c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:15.487764 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9442c-f2cf-4536-accc-0bcf85dc68c9 admin admin] PATH_INFO: `/v3/projects/273b7489aee041468a745ea43761cb02` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:15.489714 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d9442c-f2cf-4536-accc-0bcf85dc68c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=273b7489aee041468a745ea43761cb02)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:15.493854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d9442c-f2cf-4536-accc-0bcf85dc68c9 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:15.542928 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 849/1686] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:15.574547 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b09bc80-7d14-4d81-9965-93254c02d29e None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:15.577297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4b09bc80-7d14-4d81-9965-93254c02d29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:15.715593 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-87d9442c-f2cf-4536-accc-0bcf85dc68c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '273b7489aee041468a745ea43761cb02', 'request_id': 'req-87d9442c-f2cf-4536-accc-0bcf85dc68c9'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:15.756966 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:15.800950 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b09bc80-7d14-4d81-9965-93254c02d29e None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:15.855613 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-87d9442c-f2cf-4536-accc-0bcf85dc68c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '273b7489aee041468a745ea43761cb02', 'request_id': 'req-87d9442c-f2cf-4536-accc-0bcf85dc68c9'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:15.868189 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-87d9442c-f2cf-4536-accc-0bcf85dc68c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '273b7489aee041468a745ea43761cb02', 'request_id': 'req-87d9442c-f2cf-4536-accc-0bcf85dc68c9'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:15.911200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-87d9442c-f2cf-4536-accc-0bcf85dc68c9 admin admin] The token cache is being invalidate because project 273b7489aee041468a745ea43761cb02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:35:15.912579 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-87d9442c-f2cf-4536-accc-0bcf85dc68c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:15.915648 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-87d9442c-f2cf-4536-accc-0bcf85dc68c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:15.921732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 838/1687] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:35:15 2022] DELETE /identity/v3/projects/273b7489aee041468a745ea43761cb02 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:15.923442 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9442c-f2cf-4536-accc-0bcf85dc68c9 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:15.929793 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-87d9442c-f2cf-4536-accc-0bcf85dc68c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:15.988841 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62226f6-2704-494c-a59a-1aa76f483668 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:15.989561 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62226f6-2704-494c-a59a-1aa76f483668 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:15.990371 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62226f6-2704-494c-a59a-1aa76f483668 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:15.990713 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62226f6-2704-494c-a59a-1aa76f483668 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:15.998529 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b62226f6-2704-494c-a59a-1aa76f483668 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:16.147891 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9442c-f2cf-4536-accc-0bcf85dc68c9 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:16.168922 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b65031b-7cad-419a-9df3-1471e943b3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:16.169827 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65031b-7cad-419a-9df3-1471e943b3ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:16.170507 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65031b-7cad-419a-9df3-1471e943b3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:16.170920 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b65031b-7cad-419a-9df3-1471e943b3ed admin admin] PATH_INFO: `/v3/projects/5f8d42bcfc0947baa7debb6c4eb692c7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:16.175564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62226f6-2704-494c-a59a-1aa76f483668 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:16.176200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b65031b-7cad-419a-9df3-1471e943b3ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f8d42bcfc0947baa7debb6c4eb692c7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:16.177993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62226f6-2704-494c-a59a-1aa76f483668 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:16.186916 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b65031b-7cad-419a-9df3-1471e943b3ed admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:16.195204 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 850/1688] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 623 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:16.196572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a4811-ce71-4ccc-be59-84b144653252 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:16.208057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a4811-ce71-4ccc-be59-84b144653252 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:16.208057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a4811-ce71-4ccc-be59-84b144653252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:16.262781 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-3d3a4811-ce71-4ccc-be59-84b144653252 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:16.264457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3d3a4811-ce71-4ccc-be59-84b144653252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:16.293629 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 851/1689] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:16.303697 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a4811-ce71-4ccc-be59-84b144653252 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:16.312361 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3d3a4811-ce71-4ccc-be59-84b144653252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:16.336330 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a4811-ce71-4ccc-be59-84b144653252 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:16.356787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc50fb-ace0-4464-979d-b03227206f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:16.357811 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc50fb-ace0-4464-979d-b03227206f8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:16.358077 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc50fb-ace0-4464-979d-b03227206f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:16.358362 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc50fb-ace0-4464-979d-b03227206f8c admin admin] PATH_INFO: `/v3/projects/4ec6013ce82b48da8a79d818ab09bd0b` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:16.359763 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc50fb-ace0-4464-979d-b03227206f8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ec6013ce82b48da8a79d818ab09bd0b)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:16.364017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc50fb-ace0-4464-979d-b03227206f8c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:16.473853 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-4b65031b-7cad-419a-9df3-1471e943b3ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f8d42bcfc0947baa7debb6c4eb692c7', 'request_id': 'req-4b65031b-7cad-419a-9df3-1471e943b3ed'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:16.587705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-4b65031b-7cad-419a-9df3-1471e943b3ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f8d42bcfc0947baa7debb6c4eb692c7', 'request_id': 'req-4b65031b-7cad-419a-9df3-1471e943b3ed'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:16.590030 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e9fc50fb-ace0-4464-979d-b03227206f8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ec6013ce82b48da8a79d818ab09bd0b', 'request_id': 'req-e9fc50fb-ace0-4464-979d-b03227206f8c'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:16.600600 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-4b65031b-7cad-419a-9df3-1471e943b3ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f8d42bcfc0947baa7debb6c4eb692c7', 'request_id': 'req-4b65031b-7cad-419a-9df3-1471e943b3ed'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:16.630905 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-4b65031b-7cad-419a-9df3-1471e943b3ed admin admin] The token cache is being invalidate because project 5f8d42bcfc0947baa7debb6c4eb692c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:35:16.631800 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-4b65031b-7cad-419a-9df3-1471e943b3ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:16.633021 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-4b65031b-7cad-419a-9df3-1471e943b3ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:16.636498 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 839/1690] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:35:15 2022] DELETE /identity/v3/projects/5f8d42bcfc0947baa7debb6c4eb692c7 => generated 0 bytes in 714 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:16.643061 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb55012-bfaa-41c2-97e8-527e0d6ca74f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:16.643476 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb55012-bfaa-41c2-97e8-527e0d6ca74f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:16.643955 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb55012-bfaa-41c2-97e8-527e0d6ca74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:16.681890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e9fc50fb-ace0-4464-979d-b03227206f8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ec6013ce82b48da8a79d818ab09bd0b', 'request_id': 'req-e9fc50fb-ace0-4464-979d-b03227206f8c'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:16.691106 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e9fc50fb-ace0-4464-979d-b03227206f8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ec6013ce82b48da8a79d818ab09bd0b', 'request_id': 'req-e9fc50fb-ace0-4464-979d-b03227206f8c'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:16.712081 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e9fc50fb-ace0-4464-979d-b03227206f8c admin admin] The token cache is being invalidate because project 4ec6013ce82b48da8a79d818ab09bd0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:35:16.712718 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e9fc50fb-ace0-4464-979d-b03227206f8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:16.713644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e9fc50fb-ace0-4464-979d-b03227206f8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:16.715623 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 852/1691] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:35:16 2022] DELETE /identity/v3/projects/4ec6013ce82b48da8a79d818ab09bd0b => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:16.717268 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2d431-6eca-4806-821d-a37737707336 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:16.717821 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2d431-6eca-4806-821d-a37737707336 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:16.718251 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2d431-6eca-4806-821d-a37737707336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:16.734877 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-1fb55012-bfaa-41c2-97e8-527e0d6ca74f None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:16.742734 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-cad2d431-6eca-4806-821d-a37737707336 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:16.744382 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-cad2d431-6eca-4806-821d-a37737707336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:16.747214 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1fb55012-bfaa-41c2-97e8-527e0d6ca74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:16.883976 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 853/1692] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:16.889675 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2a59e-fcb6-49e8-b5ea-3a690c6056cd None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:16.890027 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2a59e-fcb6-49e8-b5ea-3a690c6056cd None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:16.890306 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea2a59e-fcb6-49e8-b5ea-3a690c6056cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:16.938156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-aea2a59e-fcb6-49e8-b5ea-3a690c6056cd None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:16.941887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-aea2a59e-fcb6-49e8-b5ea-3a690c6056cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:16.950407 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 840/1693] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:16.953173 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b29ec-f2f2-482b-acd0-2ed3efe8e0b8 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:16.953797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b29ec-f2f2-482b-acd0-2ed3efe8e0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:16.954277 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b29ec-f2f2-482b-acd0-2ed3efe8e0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:16.973549 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 854/1694] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:16.974864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2a59e-fcb6-49e8-b5ea-3a690c6056cd None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:16.986052 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-c58b29ec-f2f2-482b-acd0-2ed3efe8e0b8 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:16.988447 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c58b29ec-f2f2-482b-acd0-2ed3efe8e0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:17.002689 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 841/1695] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:17.004802 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ccaf7-aa5d-4047-981f-1de3905fc4b9 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:17.006202 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ccaf7-aa5d-4047-981f-1de3905fc4b9 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:17.006812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ccaf7-aa5d-4047-981f-1de3905fc4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:17.023136 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea2a59e-fcb6-49e8-b5ea-3a690c6056cd None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:17.033805 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-827ccaf7-aa5d-4047-981f-1de3905fc4b9 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:17.035642 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-827ccaf7-aa5d-4047-981f-1de3905fc4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:17.039747 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48ad34-33da-4a7d-a0ca-ac94c5c47707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:17.040409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48ad34-33da-4a7d-a0ca-ac94c5c47707 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:17.040698 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48ad34-33da-4a7d-a0ca-ac94c5c47707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:17.040968 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48ad34-33da-4a7d-a0ca-ac94c5c47707 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:17.046725 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 842/1696] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:17.047970 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ccaf7-aa5d-4047-981f-1de3905fc4b9 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:17.051259 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f48ad34-33da-4a7d-a0ca-ac94c5c47707 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:35:17.051643 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f48ad34-33da-4a7d-a0ca-ac94c5c47707 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:17.060855 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ccaf7-aa5d-4047-981f-1de3905fc4b9 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:17.063282 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f48ad34-33da-4a7d-a0ca-ac94c5c47707 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:17.067097 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6993c-a5ae-4025-8c26-1f5fb0476e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:17.068182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6993c-a5ae-4025-8c26-1f5fb0476e54 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:17.068620 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6993c-a5ae-4025-8c26-1f5fb0476e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:17.069022 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6993c-a5ae-4025-8c26-1f5fb0476e54 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:17.069958 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef6993c-a5ae-4025-8c26-1f5fb0476e54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:35:17.070373 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef6993c-a5ae-4025-8c26-1f5fb0476e54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:17.072269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef6993c-a5ae-4025-8c26-1f5fb0476e54 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:17.093537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 843/1697] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:35:17.093537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6993c-a5ae-4025-8c26-1f5fb0476e54 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:17.110385 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6993c-a5ae-4025-8c26-1f5fb0476e54 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:17.114027 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 855/1698] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:35:17.115593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48ad34-33da-4a7d-a0ca-ac94c5c47707 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:17.126027 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d14a2-2ff2-46f8-8481-42d7f156d593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:17.127012 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d14a2-2ff2-46f8-8481-42d7f156d593 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:17.127405 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d14a2-2ff2-46f8-8481-42d7f156d593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:17.127804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d14a2-2ff2-46f8-8481-42d7f156d593 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:17.128697 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d14a2-2ff2-46f8-8481-42d7f156d593 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:35:17.129119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d14a2-2ff2-46f8-8481-42d7f156d593 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:17.131813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45d14a2-2ff2-46f8-8481-42d7f156d593 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:17.139714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48ad34-33da-4a7d-a0ca-ac94c5c47707 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:17.153749 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466fca0d-793a-4c86-bb84-ba7ec7f77e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:17.157542 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fca0d-793a-4c86-bb84-ba7ec7f77e65 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:17.157676 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 844/1699] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:35:17.157767 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fca0d-793a-4c86-bb84-ba7ec7f77e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:17.157849 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d14a2-2ff2-46f8-8481-42d7f156d593 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:17.157931 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fca0d-793a-4c86-bb84-ba7ec7f77e65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:17.157931 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466fca0d-793a-4c86-bb84-ba7ec7f77e65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:17.167389 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466fca0d-793a-4c86-bb84-ba7ec7f77e65 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:17.180512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d14a2-2ff2-46f8-8481-42d7f156d593 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:17.188663 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f27311-80f0-40ff-8b2d-d8ee3ab67990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:17.189378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f27311-80f0-40ff-8b2d-d8ee3ab67990 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:17.189741 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f27311-80f0-40ff-8b2d-d8ee3ab67990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:17.190220 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f27311-80f0-40ff-8b2d-d8ee3ab67990 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:17.192870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f27311-80f0-40ff-8b2d-d8ee3ab67990 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:17.198020 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f27311-80f0-40ff-8b2d-d8ee3ab67990 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:17.267256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:17.271901 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 856/1700] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:17 2022] POST /identity/v3/projects => generated 409 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:17.273179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466fca0d-793a-4c86-bb84-ba7ec7f77e65 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:17.354569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 845/1701] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:17 2022] POST /identity/v3/projects => generated 363 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:17.355668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f27311-80f0-40ff-8b2d-d8ee3ab67990 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:17.373586 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:17.385130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466fca0d-793a-4c86-bb84-ba7ec7f77e65 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:17.470075 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f27311-80f0-40ff-8b2d-d8ee3ab67990 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:17.479586 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31217a87-942e-4273-b1f7-9aad1df16e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:17.480426 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31217a87-942e-4273-b1f7-9aad1df16e20 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:17.480781 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31217a87-942e-4273-b1f7-9aad1df16e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:17.481490 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31217a87-942e-4273-b1f7-9aad1df16e20 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:17.482272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31217a87-942e-4273-b1f7-9aad1df16e20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:17.485345 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31217a87-942e-4273-b1f7-9aad1df16e20 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:17.501068 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff50a11-c125-4c88-94ea-a7456e092141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:17.501962 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff50a11-c125-4c88-94ea-a7456e092141 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:17.505180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff50a11-c125-4c88-94ea-a7456e092141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:17.505180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff50a11-c125-4c88-94ea-a7456e092141 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:17.505180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff50a11-c125-4c88-94ea-a7456e092141 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:17.507642 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff50a11-c125-4c88-94ea-a7456e092141 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:17.619797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 846/1702] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:17 2022] POST /identity/v3/users => generated 305 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:17.643124 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcbe3b-2f04-4e17-90c1-cb45be38483f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:17.643322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 857/1703] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:17 2022] POST /identity/v3/projects => generated 363 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:17.643322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff50a11-c125-4c88-94ea-a7456e092141 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:17.651267 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcbe3b-2f04-4e17-90c1-cb45be38483f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:17.651912 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcbe3b-2f04-4e17-90c1-cb45be38483f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:17.718431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-87dcbe3b-2f04-4e17-90c1-cb45be38483f None None] MFA Rules not processed for user `104a135932a144269f1f2aa7359098ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:17.719733 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-87dcbe3b-2f04-4e17-90c1-cb45be38483f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:17.777109 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff50a11-c125-4c88-94ea-a7456e092141 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:17.786379 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e4432-41ce-4489-8f85-0c6e50e17d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:17.804202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e4432-41ce-4489-8f85-0c6e50e17d81 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:17.804202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e4432-41ce-4489-8f85-0c6e50e17d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:17.804202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e4432-41ce-4489-8f85-0c6e50e17d81 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:17.804202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144e4432-41ce-4489-8f85-0c6e50e17d81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:17.804202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144e4432-41ce-4489-8f85-0c6e50e17d81 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:17.863110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 847/1704] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:17.864360 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dcbe3b-2f04-4e17-90c1-cb45be38483f None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:17.904000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dcbe3b-2f04-4e17-90c1-cb45be38483f None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:17.911090 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac409f57-6173-4310-b695-34fccbecfbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:17.912830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac409f57-6173-4310-b695-34fccbecfbd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:17.913472 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac409f57-6173-4310-b695-34fccbecfbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:17.913916 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac409f57-6173-4310-b695-34fccbecfbd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:17.915017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac409f57-6173-4310-b695-34fccbecfbd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:17.917440 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac409f57-6173-4310-b695-34fccbecfbd3 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:17.937689 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 858/1705] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:17 2022] POST /identity/v3/users => generated 282 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:17.937689 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e4432-41ce-4489-8f85-0c6e50e17d81 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:17.945222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 848/1706] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:17.946234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac409f57-6173-4310-b695-34fccbecfbd3 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:17.950059 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ac409f57-6173-4310-b695-34fccbecfbd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:17.961292 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e4432-41ce-4489-8f85-0c6e50e17d81 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:17.976486 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ae49a-4cbd-4152-91c4-48df10e82037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:17.976486 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ae49a-4cbd-4152-91c4-48df10e82037 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:17.976486 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ae49a-4cbd-4152-91c4-48df10e82037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:17.976486 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ae49a-4cbd-4152-91c4-48df10e82037 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:17.976873 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7ae49a-4cbd-4152-91c4-48df10e82037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:17.976873 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7ae49a-4cbd-4152-91c4-48df10e82037 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:18.080660 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 859/1707] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:17 2022] POST /identity/v3/users => generated 282 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:18.081731 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ae49a-4cbd-4152-91c4-48df10e82037 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:18.101844 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac409f57-6173-4310-b695-34fccbecfbd3 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:18.112602 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388556b-b4dd-4a37-b3eb-0c400ca618b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:18.113279 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388556b-b4dd-4a37-b3eb-0c400ca618b9 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:18.116381 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ae49a-4cbd-4152-91c4-48df10e82037 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:18.116666 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388556b-b4dd-4a37-b3eb-0c400ca618b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:18.117215 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388556b-b4dd-4a37-b3eb-0c400ca618b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:18.128773 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99325a-fd3e-4d35-9c43-bb6f172f61a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:18.129464 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99325a-fd3e-4d35-9c43-bb6f172f61a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:18.129748 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99325a-fd3e-4d35-9c43-bb6f172f61a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:18.131382 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99325a-fd3e-4d35-9c43-bb6f172f61a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:18.132127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99325a-fd3e-4d35-9c43-bb6f172f61a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:18.137353 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99325a-fd3e-4d35-9c43-bb6f172f61a9 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:18.139075 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388556b-b4dd-4a37-b3eb-0c400ca618b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:18.140734 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388556b-b4dd-4a37-b3eb-0c400ca618b9 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:18.163354 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 849/1708] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:18.164250 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388556b-b4dd-4a37-b3eb-0c400ca618b9 service nova] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:18.171415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 860/1709] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:18.172758 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99325a-fd3e-4d35-9c43-bb6f172f61a9 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:18.201284 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99325a-fd3e-4d35-9c43-bb6f172f61a9 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:18.202634 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388556b-b4dd-4a37-b3eb-0c400ca618b9 service nova] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:18.219372 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51abb7e0-cd6b-48b9-9562-ae55fddb2eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:18.220343 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abb7e0-cd6b-48b9-9562-ae55fddb2eed admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:18.220789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abb7e0-cd6b-48b9-9562-ae55fddb2eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:18.221220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abb7e0-cd6b-48b9-9562-ae55fddb2eed admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:18.222173 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51abb7e0-cd6b-48b9-9562-ae55fddb2eed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:18.225523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51abb7e0-cd6b-48b9-9562-ae55fddb2eed admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:18.228199 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80842b6e-11f5-4bb7-99ff-b0b10911933f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:18.228965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80842b6e-11f5-4bb7-99ff-b0b10911933f admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:18.229259 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80842b6e-11f5-4bb7-99ff-b0b10911933f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:18.230132 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80842b6e-11f5-4bb7-99ff-b0b10911933f admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:18.230495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80842b6e-11f5-4bb7-99ff-b0b10911933f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:18.232830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80842b6e-11f5-4bb7-99ff-b0b10911933f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:18.262725 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 850/1710] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:18.266276 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80842b6e-11f5-4bb7-99ff-b0b10911933f admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:18.279165 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 861/1711] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:18.282005 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80842b6e-11f5-4bb7-99ff-b0b10911933f admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:18.282327 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51abb7e0-cd6b-48b9-9562-ae55fddb2eed admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:18.290124 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ddd7ff-60a3-4ef9-8cc1-6c3678291ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:18.293263 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddd7ff-60a3-4ef9-8cc1-6c3678291ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:18.293263 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddd7ff-60a3-4ef9-8cc1-6c3678291ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:18.293263 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddd7ff-60a3-4ef9-8cc1-6c3678291ef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:18.293263 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ddd7ff-60a3-4ef9-8cc1-6c3678291ef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:18.295271 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ddd7ff-60a3-4ef9-8cc1-6c3678291ef1 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:18.322068 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51abb7e0-cd6b-48b9-9562-ae55fddb2eed admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:18.328000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 851/1712] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:18.330999 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ddd7ff-60a3-4ef9-8cc1-6c3678291ef1 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:18.344731 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed99d9-1a49-4c5a-b9e3-0471f5e814ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:18.345657 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed99d9-1a49-4c5a-b9e3-0471f5e814ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:18.345948 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed99d9-1a49-4c5a-b9e3-0471f5e814ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:18.346228 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed99d9-1a49-4c5a-b9e3-0471f5e814ea admin admin] PATH_INFO: `/v3/projects/96d122bd8bbf4048ae9918321f065a7f/users/4d70af94c5874888b86750447f2c5fbd/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:18.352758 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:18.381991 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ddd7ff-60a3-4ef9-8cc1-6c3678291ef1 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:18.400794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087008a-97d2-443e-9335-13d8b066540e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:18.403743 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed99d9-1a49-4c5a-b9e3-0471f5e814ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=96d122bd8bbf4048ae9918321f065a7f, user_id=4d70af94c5874888b86750447f2c5fbd, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:18.404423 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087008a-97d2-443e-9335-13d8b066540e admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:18.407177 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087008a-97d2-443e-9335-13d8b066540e admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:18.408076 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087008a-97d2-443e-9335-13d8b066540e admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:18.408893 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5087008a-97d2-443e-9335-13d8b066540e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:18.415559 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed99d9-1a49-4c5a-b9e3-0471f5e814ea admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:18.419883 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5087008a-97d2-443e-9335-13d8b066540e admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:18.458960 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 852/1713] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:18.459543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087008a-97d2-443e-9335-13d8b066540e admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:18.494416 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087008a-97d2-443e-9335-13d8b066540e admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:18.503822 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d302a7cd-c50c-4227-862e-b382cd3fa430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:18.505033 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302a7cd-c50c-4227-862e-b382cd3fa430 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:18.509842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302a7cd-c50c-4227-862e-b382cd3fa430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:18.509842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302a7cd-c50c-4227-862e-b382cd3fa430 admin admin] PATH_INFO: `/v3/projects/440a5550bef3447eaae6b5f1a3674116/users/ed8ef1333d484db284ffb6e9498bcaec/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:18.510141 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 862/1714] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:18 2022] PUT /identity/v3/projects/96d122bd8bbf4048ae9918321f065a7f/users/4d70af94c5874888b86750447f2c5fbd/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:18.510141 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed99d9-1a49-4c5a-b9e3-0471f5e814ea admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:18.561365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d302a7cd-c50c-4227-862e-b382cd3fa430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=440a5550bef3447eaae6b5f1a3674116, user_id=ed8ef1333d484db284ffb6e9498bcaec, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:18.564663 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d302a7cd-c50c-4227-862e-b382cd3fa430 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:18.622493 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed99d9-1a49-4c5a-b9e3-0471f5e814ea admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:18.629933 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21338bb5-0074-4eee-9a30-4773e051659c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:18.631014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21338bb5-0074-4eee-9a30-4773e051659c admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:18.631298 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21338bb5-0074-4eee-9a30-4773e051659c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:18.631579 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21338bb5-0074-4eee-9a30-4773e051659c admin admin] PATH_INFO: `/v3/projects/ab2b8ea6e01c4ef3ab335de16a9eb842/users/8e73d92c276b4fd68bc13d784446c3fb/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:18.645761 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 853/1715] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:18 2022] PUT /identity/v3/projects/440a5550bef3447eaae6b5f1a3674116/users/ed8ef1333d484db284ffb6e9498bcaec/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:18.648437 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ade3d-11e6-4af9-be76-6ded212705dc None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:18.648732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ade3d-11e6-4af9-be76-6ded212705dc None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:18.649013 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ade3d-11e6-4af9-be76-6ded212705dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:18.658479 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21338bb5-0074-4eee-9a30-4773e051659c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab2b8ea6e01c4ef3ab335de16a9eb842, user_id=8e73d92c276b4fd68bc13d784446c3fb, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:18.667564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21338bb5-0074-4eee-9a30-4773e051659c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:18.700009 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-2a1ade3d-11e6-4af9-be76-6ded212705dc None None] MFA Rules not processed for user `4d70af94c5874888b86750447f2c5fbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:18.701411 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-2a1ade3d-11e6-4af9-be76-6ded212705dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:18.782280 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 863/1716] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:18 2022] PUT /identity/v3/projects/ab2b8ea6e01c4ef3ab335de16a9eb842/users/8e73d92c276b4fd68bc13d784446c3fb/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:18.785030 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bf1be-a5d6-43c0-badb-e04dd0521d8d None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:18.785030 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bf1be-a5d6-43c0-badb-e04dd0521d8d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:18.785030 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bf1be-a5d6-43c0-badb-e04dd0521d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:18.789693 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 854/1717] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:18 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:18.791779 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0be55f-2ebf-409e-858e-7ec1a680805a None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:18.792175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0be55f-2ebf-409e-858e-7ec1a680805a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:18.792545 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0be55f-2ebf-409e-858e-7ec1a680805a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:18.825463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-0c0be55f-2ebf-409e-858e-7ec1a680805a None None] MFA Rules not processed for user `ed8ef1333d484db284ffb6e9498bcaec`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:18.828033 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-0c0be55f-2ebf-409e-858e-7ec1a680805a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:18.857151 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-042bf1be-a5d6-43c0-badb-e04dd0521d8d None None] MFA Rules not processed for user `dac1130950b8457ea3a9349346437357`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:18.859137 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-042bf1be-a5d6-43c0-badb-e04dd0521d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:18.942277 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:18.952172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 855/1718] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:18 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:18.955343 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34393693-f61a-43e8-9bb0-09ebc206052e None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:18.955949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34393693-f61a-43e8-9bb0-09ebc206052e None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:18.956531 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34393693-f61a-43e8-9bb0-09ebc206052e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:18.976055 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 864/1719] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:18.977937 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47e674-d53b-441a-a0f1-2bdc6505f027 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:18.977937 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47e674-d53b-441a-a0f1-2bdc6505f027 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:18.978089 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47e674-d53b-441a-a0f1-2bdc6505f027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:19.008122 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-da47e674-d53b-441a-a0f1-2bdc6505f027 None None] MFA Rules not processed for user `8e73d92c276b4fd68bc13d784446c3fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:19.009189 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-da47e674-d53b-441a-a0f1-2bdc6505f027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:19.027123 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-34393693-f61a-43e8-9bb0-09ebc206052e None None] MFA Rules not processed for user `402ec1b74c464319a5d4a5628f6f9e2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:19.028673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-34393693-f61a-43e8-9bb0-09ebc206052e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:19.121092 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 865/1720] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:18 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:19.121999 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47e674-d53b-441a-a0f1-2bdc6505f027 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:19.189317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 856/1721] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:19.191488 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34393693-f61a-43e8-9bb0-09ebc206052e None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:19.255050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47e674-d53b-441a-a0f1-2bdc6505f027 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:19.278022 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9801965b-0982-40ee-8991-9306d18b2d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:19.278795 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801965b-0982-40ee-8991-9306d18b2d41 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:19.279650 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801965b-0982-40ee-8991-9306d18b2d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:19.280014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801965b-0982-40ee-8991-9306d18b2d41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:19.280784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9801965b-0982-40ee-8991-9306d18b2d41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:19.281402 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34393693-f61a-43e8-9bb0-09ebc206052e None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:19.289421 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b291ea-7564-4cfb-b94c-1570115b58eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:19.290195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b291ea-7564-4cfb-b94c-1570115b58eb admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:19.290481 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b291ea-7564-4cfb-b94c-1570115b58eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:19.292979 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b291ea-7564-4cfb-b94c-1570115b58eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:19.292979 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b291ea-7564-4cfb-b94c-1570115b58eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:19.294617 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b291ea-7564-4cfb-b94c-1570115b58eb admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:19.297346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9801965b-0982-40ee-8991-9306d18b2d41 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:19.417589 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 857/1722] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:19 2022] POST /identity/v3/projects => generated 363 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:19.417589 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b291ea-7564-4cfb-b94c-1570115b58eb admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:19.420703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d3b291ea-7564-4cfb-b94c-1570115b58eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:19.434770 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 866/1723] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:19 2022] POST /identity/v3/projects => generated 407 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:19.437738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9801965b-0982-40ee-8991-9306d18b2d41 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:19.552034 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9801965b-0982-40ee-8991-9306d18b2d41 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:19.569717 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b291ea-7564-4cfb-b94c-1570115b58eb admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:19.579182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f9b31-efab-4ee9-adc9-a27e387d4c09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:19.579850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f9b31-efab-4ee9-adc9-a27e387d4c09 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:19.580174 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f9b31-efab-4ee9-adc9-a27e387d4c09 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:19.581009 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f9b31-efab-4ee9-adc9-a27e387d4c09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:19.625689 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d06e6e-83eb-49a9-9f7b-cec1c8a3fa5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:19.626367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d06e6e-83eb-49a9-9f7b-cec1c8a3fa5c admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:19.626652 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d06e6e-83eb-49a9-9f7b-cec1c8a3fa5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:19.627995 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d06e6e-83eb-49a9-9f7b-cec1c8a3fa5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:19.628826 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d06e6e-83eb-49a9-9f7b-cec1c8a3fa5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:19.631902 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d06e6e-83eb-49a9-9f7b-cec1c8a3fa5c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:19.696035 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f9b31-efab-4ee9-adc9-a27e387d4c09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:19.699387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25f9b31-efab-4ee9-adc9-a27e387d4c09 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:19.741837 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 858/1724] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:19.743038 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f9b31-efab-4ee9-adc9-a27e387d4c09 service nova] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:19.754143 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a25f9b31-efab-4ee9-adc9-a27e387d4c09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:19.794755 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 867/1725] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:19 2022] POST /identity/v3/projects => generated 365 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:19.797114 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d06e6e-83eb-49a9-9f7b-cec1c8a3fa5c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:19.800346 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f9b31-efab-4ee9-adc9-a27e387d4c09 service nova] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:19.935192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d06e6e-83eb-49a9-9f7b-cec1c8a3fa5c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:19.936433 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7899fdf-be74-443e-9602-6ac7fe801acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:19.937289 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7899fdf-be74-443e-9602-6ac7fe801acb admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:19.937571 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7899fdf-be74-443e-9602-6ac7fe801acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:19.937855 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7899fdf-be74-443e-9602-6ac7fe801acb admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:19.938516 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7899fdf-be74-443e-9602-6ac7fe801acb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:19.940636 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7899fdf-be74-443e-9602-6ac7fe801acb admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:19.943950 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bcc9b8-0559-4e61-8cba-37bf52d20163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:19.944509 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bcc9b8-0559-4e61-8cba-37bf52d20163 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:19.944853 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bcc9b8-0559-4e61-8cba-37bf52d20163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:19.945149 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bcc9b8-0559-4e61-8cba-37bf52d20163 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:19.945885 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bcc9b8-0559-4e61-8cba-37bf52d20163 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:19.948084 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bcc9b8-0559-4e61-8cba-37bf52d20163 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:20.043752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 868/1726] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:19 2022] POST /identity/v3/users => generated 288 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:20.044498 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bcc9b8-0559-4e61-8cba-37bf52d20163 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:20.077538 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bcc9b8-0559-4e61-8cba-37bf52d20163 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:20.097402 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a6b94-325c-46e5-929f-2b0eb6efa91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:20.098731 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a6b94-325c-46e5-929f-2b0eb6efa91e admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:20.099778 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a6b94-325c-46e5-929f-2b0eb6efa91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:20.100157 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a6b94-325c-46e5-929f-2b0eb6efa91e admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:20.101138 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a6b94-325c-46e5-929f-2b0eb6efa91e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:20.104633 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a6b94-325c-46e5-929f-2b0eb6efa91e admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:20.108328 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 859/1727] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:19 2022] POST /identity/v3/projects => generated 405 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:20.109209 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7899fdf-be74-443e-9602-6ac7fe801acb admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:20.121869 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:20.214229 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 869/1728] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:20 2022] POST /identity/v3/users => generated 310 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:20.219479 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a6b94-325c-46e5-929f-2b0eb6efa91e admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:20.244909 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a6b94-325c-46e5-929f-2b0eb6efa91e admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:20.259729 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec46ea5-4933-408b-9962-9dc5cb9046b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:20.260410 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec46ea5-4933-408b-9962-9dc5cb9046b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:20.260701 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec46ea5-4933-408b-9962-9dc5cb9046b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:20.260981 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec46ea5-4933-408b-9962-9dc5cb9046b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:20.261726 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec46ea5-4933-408b-9962-9dc5cb9046b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:20.263453 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec46ea5-4933-408b-9962-9dc5cb9046b7 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:20.300251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7899fdf-be74-443e-9602-6ac7fe801acb admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:20.300736 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 870/1729] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:20.300736 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec46ea5-4933-408b-9962-9dc5cb9046b7 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:20.304506 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c312c3b-ce14-48b3-856b-e8ab2374a182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:20.305376 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c312c3b-ce14-48b3-856b-e8ab2374a182 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:20.305757 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c312c3b-ce14-48b3-856b-e8ab2374a182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:20.306996 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c312c3b-ce14-48b3-856b-e8ab2374a182 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:20.307824 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c312c3b-ce14-48b3-856b-e8ab2374a182 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:20.314436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec46ea5-4933-408b-9962-9dc5cb9046b7 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:20.315602 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c312c3b-ce14-48b3-856b-e8ab2374a182 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:20.320665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d75470-cd00-4ca6-9785-7e2b8aab23a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:20.322958 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d75470-cd00-4ca6-9785-7e2b8aab23a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:20.323453 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d75470-cd00-4ca6-9785-7e2b8aab23a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:20.324309 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d75470-cd00-4ca6-9785-7e2b8aab23a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:20.325471 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d75470-cd00-4ca6-9785-7e2b8aab23a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:20.327745 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d75470-cd00-4ca6-9785-7e2b8aab23a6 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:20.455896 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 860/1730] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:20 2022] POST /identity/v3/users => generated 289 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:20.456872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c312c3b-ce14-48b3-856b-e8ab2374a182 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:20.459686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 871/1731] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:20 2022] POST /identity/v3/users => generated 309 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:20.460689 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d75470-cd00-4ca6-9785-7e2b8aab23a6 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:20.473317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d75470-cd00-4ca6-9785-7e2b8aab23a6 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:20.478072 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c312c3b-ce14-48b3-856b-e8ab2374a182 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:20.481575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1dcd3b-782d-4b67-9aa8-9517dfcd323c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:20.482595 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1dcd3b-782d-4b67-9aa8-9517dfcd323c admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:20.482928 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1dcd3b-782d-4b67-9aa8-9517dfcd323c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:20.483371 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1dcd3b-782d-4b67-9aa8-9517dfcd323c admin admin] PATH_INFO: `/v3/projects/e007b1b172b64559be6ada4d6b1a4f33/users/e600d24c9e0442538f457dcccd52f581/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:20.491254 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a4a5f-2a6e-480e-9a5e-208c61ec1146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:20.492046 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a4a5f-2a6e-480e-9a5e-208c61ec1146 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:20.493271 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a4a5f-2a6e-480e-9a5e-208c61ec1146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:20.494530 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a4a5f-2a6e-480e-9a5e-208c61ec1146 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:20.495810 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a4a5f-2a6e-480e-9a5e-208c61ec1146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:20.499176 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a4a5f-2a6e-480e-9a5e-208c61ec1146 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:20.529058 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1dcd3b-782d-4b67-9aa8-9517dfcd323c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e007b1b172b64559be6ada4d6b1a4f33, user_id=e600d24c9e0442538f457dcccd52f581, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:20.535953 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1dcd3b-782d-4b67-9aa8-9517dfcd323c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:20.537944 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 861/1732] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:20.540354 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a4a5f-2a6e-480e-9a5e-208c61ec1146 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:20.570683 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a4a5f-2a6e-480e-9a5e-208c61ec1146 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:20.578690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0644a04-0b5b-43df-a71b-b9945ce533a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:20.579424 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0644a04-0b5b-43df-a71b-b9945ce533a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:20.579762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0644a04-0b5b-43df-a71b-b9945ce533a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:20.580723 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0644a04-0b5b-43df-a71b-b9945ce533a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:20.581529 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0644a04-0b5b-43df-a71b-b9945ce533a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:20.588022 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0644a04-0b5b-43df-a71b-b9945ce533a5 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:20.612161 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 862/1733] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:20.614284 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0644a04-0b5b-43df-a71b-b9945ce533a5 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:20.643194 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0644a04-0b5b-43df-a71b-b9945ce533a5 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:20.653151 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 872/1734] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:20 2022] PUT /identity/v3/projects/e007b1b172b64559be6ada4d6b1a4f33/users/e600d24c9e0442538f457dcccd52f581/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:20.657751 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1dcd3b-782d-4b67-9aa8-9517dfcd323c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:20.822363 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcead1-a9dc-46d5-9336-e6b6ccafdc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:20.823122 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcead1-a9dc-46d5-9336-e6b6ccafdc38 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:20.823408 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcead1-a9dc-46d5-9336-e6b6ccafdc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:20.823952 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fcead1-a9dc-46d5-9336-e6b6ccafdc38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:20.824742 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fcead1-a9dc-46d5-9336-e6b6ccafdc38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:20.826420 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fcead1-a9dc-46d5-9336-e6b6ccafdc38 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:20.849838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 863/1735] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:20.850854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcead1-a9dc-46d5-9336-e6b6ccafdc38 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:20.852933 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1dcd3b-782d-4b67-9aa8-9517dfcd323c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:20.858801 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13167e-7333-4e01-b99a-878ea68e3a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:20.859984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13167e-7333-4e01-b99a-878ea68e3a6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:20.860839 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13167e-7333-4e01-b99a-878ea68e3a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:20.861288 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13167e-7333-4e01-b99a-878ea68e3a6f admin admin] PATH_INFO: `/v3/projects/d16cb313c2d84c679864a909c3b3984e/users/ac7a2382334e4623a21e8568cfea95fb/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:20.882500 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fcead1-a9dc-46d5-9336-e6b6ccafdc38 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:20.895297 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf5e4e-1378-4c0e-84b9-dcda869bb6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:20.898275 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf5e4e-1378-4c0e-84b9-dcda869bb6c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:20.903038 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf5e4e-1378-4c0e-84b9-dcda869bb6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:20.903038 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf5e4e-1378-4c0e-84b9-dcda869bb6c6 admin admin] PATH_INFO: `/v3/projects/ed022ccdd12a451288d7ee5d58c569a9/users/500365dfa78b4525b292c24a2678773e/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:20.908317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13167e-7333-4e01-b99a-878ea68e3a6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d16cb313c2d84c679864a909c3b3984e, user_id=ac7a2382334e4623a21e8568cfea95fb, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:20.912613 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13167e-7333-4e01-b99a-878ea68e3a6f admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:20.944349 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf5e4e-1378-4c0e-84b9-dcda869bb6c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed022ccdd12a451288d7ee5d58c569a9, user_id=500365dfa78b4525b292c24a2678773e, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:20.947250 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bf5e4e-1378-4c0e-84b9-dcda869bb6c6 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:20.954869 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:20.996215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 873/1736] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:20 2022] PUT /identity/v3/projects/d16cb313c2d84c679864a909c3b3984e/users/ac7a2382334e4623a21e8568cfea95fb/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:20.996215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13167e-7333-4e01-b99a-878ea68e3a6f admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:21.055016 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 864/1737] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:20 2022] PUT /identity/v3/projects/ed022ccdd12a451288d7ee5d58c569a9/users/500365dfa78b4525b292c24a2678773e/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:21.056178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf5e4e-1378-4c0e-84b9-dcda869bb6c6 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:21.157483 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13167e-7333-4e01-b99a-878ea68e3a6f admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:21.170273 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf5e4e-1378-4c0e-84b9-dcda869bb6c6 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:21.171926 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d5216-fe1a-43db-8191-1279b3017279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:21.172616 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d5216-fe1a-43db-8191-1279b3017279 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:21.172900 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d5216-fe1a-43db-8191-1279b3017279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:21.173794 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d5216-fe1a-43db-8191-1279b3017279 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:21.184966 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d5216-fe1a-43db-8191-1279b3017279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:21.184966 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d5216-fe1a-43db-8191-1279b3017279 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:21.185234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4edb6a7-4c24-494b-b784-7bc653b108a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:21.185234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edb6a7-4c24-494b-b784-7bc653b108a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:21.185234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edb6a7-4c24-494b-b784-7bc653b108a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:21.185234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edb6a7-4c24-494b-b784-7bc653b108a9 admin admin] PATH_INFO: `/v3/projects/b8e7feb2c46c45cba6797fe2d4af3993/users/a40b8502f9634a47989c94810a34d15b/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:21.207234 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 874/1738] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:21.211134 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d5216-fe1a-43db-8191-1279b3017279 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:21.219057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4edb6a7-4c24-494b-b784-7bc653b108a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8e7feb2c46c45cba6797fe2d4af3993, user_id=a40b8502f9634a47989c94810a34d15b, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:21.225879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4edb6a7-4c24-494b-b784-7bc653b108a9 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:21.233441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d5216-fe1a-43db-8191-1279b3017279 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:21.247747 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a01f0-7c8b-4e22-afb2-a4a92ca217c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:21.248640 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a01f0-7c8b-4e22-afb2-a4a92ca217c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:21.249116 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a01f0-7c8b-4e22-afb2-a4a92ca217c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:21.249530 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a01f0-7c8b-4e22-afb2-a4a92ca217c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:21.250401 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a01f0-7c8b-4e22-afb2-a4a92ca217c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:21.253409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a01f0-7c8b-4e22-afb2-a4a92ca217c0 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:21.276271 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 875/1739] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:21.277454 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a01f0-7c8b-4e22-afb2-a4a92ca217c0 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:21.308853 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a01f0-7c8b-4e22-afb2-a4a92ca217c0 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:21.316879 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c08672-bba5-4920-8275-2073cc9007c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:21.318686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c08672-bba5-4920-8275-2073cc9007c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:21.319081 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c08672-bba5-4920-8275-2073cc9007c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:21.319406 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c08672-bba5-4920-8275-2073cc9007c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:21.320821 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c08672-bba5-4920-8275-2073cc9007c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:21.323170 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c08672-bba5-4920-8275-2073cc9007c6 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:21.353835 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 865/1740] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:21 2022] PUT /identity/v3/projects/b8e7feb2c46c45cba6797fe2d4af3993/users/a40b8502f9634a47989c94810a34d15b/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:21.354980 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4edb6a7-4c24-494b-b784-7bc653b108a9 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:21.363916 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 876/1741] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:21.365980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c08672-bba5-4920-8275-2073cc9007c6 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:21.388229 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c08672-bba5-4920-8275-2073cc9007c6 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:21.527685 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd8f80d-f31f-4bbe-9d35-c2082251c057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:21.528817 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8f80d-f31f-4bbe-9d35-c2082251c057 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:21.528928 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8f80d-f31f-4bbe-9d35-c2082251c057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:21.529129 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd8f80d-f31f-4bbe-9d35-c2082251c057 admin admin] PATH_INFO: `/v3/projects/d16cb313c2d84c679864a909c3b3984e/users/ac7a2382334e4623a21e8568cfea95fb/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:21.533401 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd8f80d-f31f-4bbe-9d35-c2082251c057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d16cb313c2d84c679864a909c3b3984e, user_id=ac7a2382334e4623a21e8568cfea95fb, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:21.537454 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd8f80d-f31f-4bbe-9d35-c2082251c057 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:21.554725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 877/1742] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:21 2022] PUT /identity/v3/projects/d16cb313c2d84c679864a909c3b3984e/users/ac7a2382334e4623a21e8568cfea95fb/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:21.556426 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd8f80d-f31f-4bbe-9d35-c2082251c057 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:21.566245 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:21.614476 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4edb6a7-4c24-494b-b784-7bc653b108a9 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:21.652654 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd8f80d-f31f-4bbe-9d35-c2082251c057 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:21.659884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b88c67-37e6-4df4-9416-c8a0646400ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:21.660646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b88c67-37e6-4df4-9416-c8a0646400ea admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:21.660928 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b88c67-37e6-4df4-9416-c8a0646400ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:21.661276 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b88c67-37e6-4df4-9416-c8a0646400ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:21.664272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b88c67-37e6-4df4-9416-c8a0646400ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:21.666476 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b88c67-37e6-4df4-9416-c8a0646400ea admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:21.680266 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 878/1743] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:21.681058 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b88c67-37e6-4df4-9416-c8a0646400ea admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:21.697589 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b88c67-37e6-4df4-9416-c8a0646400ea admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:21.698260 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877c227-eff9-4295-9125-045ad25203d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:21.699020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877c227-eff9-4295-9125-045ad25203d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:21.699307 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877c227-eff9-4295-9125-045ad25203d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:21.699584 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877c227-eff9-4295-9125-045ad25203d8 admin admin] PATH_INFO: `/v3/projects/ed022ccdd12a451288d7ee5d58c569a9/users/500365dfa78b4525b292c24a2678773e/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:21.702404 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1877c227-eff9-4295-9125-045ad25203d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed022ccdd12a451288d7ee5d58c569a9, user_id=500365dfa78b4525b292c24a2678773e, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:21.707384 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1877c227-eff9-4295-9125-045ad25203d8 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:21.734430 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 879/1744] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:21 2022] PUT /identity/v3/projects/ed022ccdd12a451288d7ee5d58c569a9/users/500365dfa78b4525b292c24a2678773e/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:21.734430 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5186f-d95b-495c-8a3b-c7b9c959c1b3 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:21.734430 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5186f-d95b-495c-8a3b-c7b9c959c1b3 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:21.734430 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5186f-d95b-495c-8a3b-c7b9c959c1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:21.785398 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-2fa5186f-d95b-495c-8a3b-c7b9c959c1b3 None None] MFA Rules not processed for user `ac7a2382334e4623a21e8568cfea95fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:21.797268 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-2fa5186f-d95b-495c-8a3b-c7b9c959c1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:21.831780 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd2168-bd8e-4596-9802-ac83db46eacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:21.832580 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd2168-bd8e-4596-9802-ac83db46eacf admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:21.832881 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd2168-bd8e-4596-9802-ac83db46eacf admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:21.833161 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd2168-bd8e-4596-9802-ac83db46eacf admin admin] PATH_INFO: `/v3/projects/e007b1b172b64559be6ada4d6b1a4f33/users/e600d24c9e0442538f457dcccd52f581/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:21.839348 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd2168-bd8e-4596-9802-ac83db46eacf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e007b1b172b64559be6ada4d6b1a4f33, user_id=e600d24c9e0442538f457dcccd52f581, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:21.844391 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd2168-bd8e-4596-9802-ac83db46eacf admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:21.893515 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 866/1745] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:21 2022] PUT /identity/v3/projects/e007b1b172b64559be6ada4d6b1a4f33/users/e600d24c9e0442538f457dcccd52f581/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:21.895382 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd2168-bd8e-4596-9802-ac83db46eacf admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:21.970774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 880/1746] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:21 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:21.972449 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7f827-9ab3-4af7-bcdd-474e960082e1 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:21.972572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7f827-9ab3-4af7-bcdd-474e960082e1 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:21.973226 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7f827-9ab3-4af7-bcdd-474e960082e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:22.000128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-96c7f827-9ab3-4af7-bcdd-474e960082e1 None None] MFA Rules not processed for user `07c9688ca035484ba5341859574c2799`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:22.001447 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-96c7f827-9ab3-4af7-bcdd-474e960082e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:22.012243 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd2168-bd8e-4596-9802-ac83db46eacf admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:22.023522 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf5d49-290d-4867-82fc-efd9946e45f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:22.024676 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf5d49-290d-4867-82fc-efd9946e45f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:22.025100 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf5d49-290d-4867-82fc-efd9946e45f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:22.025542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf5d49-290d-4867-82fc-efd9946e45f4 admin admin] PATH_INFO: `/v3/projects/b8e7feb2c46c45cba6797fe2d4af3993/users/a40b8502f9634a47989c94810a34d15b/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:22.032208 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf5d49-290d-4867-82fc-efd9946e45f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8e7feb2c46c45cba6797fe2d4af3993, user_id=a40b8502f9634a47989c94810a34d15b, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:22.034521 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf5d49-290d-4867-82fc-efd9946e45f4 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:22.064809 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 867/1747] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:21 2022] PUT /identity/v3/projects/b8e7feb2c46c45cba6797fe2d4af3993/users/a40b8502f9634a47989c94810a34d15b/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:22.065824 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e27c25-f9ff-4e28-a3ba-8bd67a74c655 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:22.066124 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e27c25-f9ff-4e28-a3ba-8bd67a74c655 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:22.066394 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e27c25-f9ff-4e28-a3ba-8bd67a74c655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:22.135835 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-27e27c25-f9ff-4e28-a3ba-8bd67a74c655 None None] MFA Rules not processed for user `500365dfa78b4525b292c24a2678773e`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:22.138774 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-27e27c25-f9ff-4e28-a3ba-8bd67a74c655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:22.149932 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 881/1748] 10.222.0.24 () {58 vars in 1009 bytes} [Fri May 27 03:35:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:22.156458 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d6943-200a-4830-b09b-10e910518cf1 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:22.159309 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d6943-200a-4830-b09b-10e910518cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:22.159309 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d6943-200a-4830-b09b-10e910518cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:22.220273 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-8c8d6943-200a-4830-b09b-10e910518cf1 None None] MFA Rules not processed for user `e600d24c9e0442538f457dcccd52f581`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:22.221705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8c8d6943-200a-4830-b09b-10e910518cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:22.349999 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 868/1749] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:22 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:22.354017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e27c25-f9ff-4e28-a3ba-8bd67a74c655 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:22.355613 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-27e27c25-f9ff-4e28-a3ba-8bd67a74c655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:22.381982 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 882/1750] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:22 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:22.391743 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d6943-200a-4830-b09b-10e910518cf1 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:22.393730 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8c8d6943-200a-4830-b09b-10e910518cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:22.446831 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e27c25-f9ff-4e28-a3ba-8bd67a74c655 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:22.452459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409a523-e0b6-4665-b485-5ca0737c4888 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:22.453337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409a523-e0b6-4665-b485-5ca0737c4888 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:22.453758 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409a523-e0b6-4665-b485-5ca0737c4888 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:22.454195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409a523-e0b6-4665-b485-5ca0737c4888 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:22.522625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d6943-200a-4830-b09b-10e910518cf1 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:22.541238 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d222e2-796f-4192-8cb3-d6d0ff0655b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:22.542614 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d222e2-796f-4192-8cb3-d6d0ff0655b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:22.543063 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d222e2-796f-4192-8cb3-d6d0ff0655b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:22.543477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d222e2-796f-4192-8cb3-d6d0ff0655b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:22.571952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d222e2-796f-4192-8cb3-d6d0ff0655b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:22.575669 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d222e2-796f-4192-8cb3-d6d0ff0655b3 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:22.608176 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 883/1751] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:22 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:22.615438 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1409a523-e0b6-4665-b485-5ca0737c4888 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:22.615438 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1409a523-e0b6-4665-b485-5ca0737c4888 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:22.636909 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 869/1752] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:22 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:22.689792 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d222e2-796f-4192-8cb3-d6d0ff0655b3 service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:22.695376 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-47d222e2-796f-4192-8cb3-d6d0ff0655b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:22.697101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409a523-e0b6-4665-b485-5ca0737c4888 service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:22.711853 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1409a523-e0b6-4665-b485-5ca0737c4888 service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:22.718219 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fe6fb-9ce4-4357-9247-2b07d96f42e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:22.718900 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fe6fb-9ce4-4357-9247-2b07d96f42e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:22.721339 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fe6fb-9ce4-4357-9247-2b07d96f42e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:22.721339 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fe6fb-9ce4-4357-9247-2b07d96f42e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:22.737409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29fe6fb-9ce4-4357-9247-2b07d96f42e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:22.739028 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29fe6fb-9ce4-4357-9247-2b07d96f42e3 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:22.787679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 870/1753] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:22 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:22.842801 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d222e2-796f-4192-8cb3-d6d0ff0655b3 service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:22.853292 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01fbf6-775c-4067-be58-8faf5baf3e48 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d559cd7127490da84185f20684929d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:22.854337 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01fbf6-775c-4067-be58-8faf5baf3e48 service swift] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:22.858669 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01fbf6-775c-4067-be58-8faf5baf3e48 service swift] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:22.858669 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01fbf6-775c-4067-be58-8faf5baf3e48 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:22.883630 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c2fc8-fdc9-456f-a6a8-9a185949219d None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:22.884180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c2fc8-fdc9-456f-a6a8-9a185949219d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:22.884180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c2fc8-fdc9-456f-a6a8-9a185949219d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:22.943604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-5f7c2fc8-fdc9-456f-a6a8-9a185949219d None None] MFA Rules not processed for user `104a135932a144269f1f2aa7359098ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:22.947786 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-5f7c2fc8-fdc9-456f-a6a8-9a185949219d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:23.007785 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec01fbf6-775c-4067-be58-8faf5baf3e48 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:23.009413 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec01fbf6-775c-4067-be58-8faf5baf3e48 service swift] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:23.043763 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 884/1754] 10.222.0.24 () {60 vars in 1384 bytes} [Fri May 27 03:35:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:35:23.084278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 871/1755] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:23.221593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01fbf6-775c-4067-be58-8faf5baf3e48 service swift] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:23.238736 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01fbf6-775c-4067-be58-8faf5baf3e48 service swift] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:23.259842 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5807d-b5f9-4daf-92d1-22e08c002297 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:23.260520 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5807d-b5f9-4daf-92d1-22e08c002297 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:23.260836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5807d-b5f9-4daf-92d1-22e08c002297 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:23.261180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5807d-b5f9-4daf-92d1-22e08c002297 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:23.323244 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c2fc8-fdc9-456f-a6a8-9a185949219d None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:23.360790 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c2fc8-fdc9-456f-a6a8-9a185949219d None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:23.373155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6dab0c-bdbf-47c8-a89b-b4522ed61d96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:23.379020 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6dab0c-bdbf-47c8-a89b-b4522ed61d96 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:23.379595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6dab0c-bdbf-47c8-a89b-b4522ed61d96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:23.380274 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6dab0c-bdbf-47c8-a89b-b4522ed61d96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:23.424102 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:23.467151 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5807d-b5f9-4daf-92d1-22e08c002297 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:23.468637 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5807d-b5f9-4daf-92d1-22e08c002297 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:23.514024 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 885/1756] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:23 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:23.590674 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6dab0c-bdbf-47c8-a89b-b4522ed61d96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:23.592214 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6dab0c-bdbf-47c8-a89b-b4522ed61d96 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:23.601461 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:23.638946 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 872/1757] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:23 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:24.162620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5807d-b5f9-4daf-92d1-22e08c002297 service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:24.178257 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5807d-b5f9-4daf-92d1-22e08c002297 service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:24.192211 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f65aa00-c9cd-493a-bae1-815166bf45b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:24.192925 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65aa00-c9cd-493a-bae1-815166bf45b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:24.198989 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65aa00-c9cd-493a-bae1-815166bf45b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:24.204642 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f65aa00-c9cd-493a-bae1-815166bf45b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:24.430043 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f65aa00-c9cd-493a-bae1-815166bf45b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:24.430043 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f65aa00-c9cd-493a-bae1-815166bf45b8 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:24.577718 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 886/1758] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:24 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:24.770057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a1e88-155c-47e1-a46a-71ed333f2eaf None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:24.770555 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a1e88-155c-47e1-a46a-71ed333f2eaf None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:24.770649 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a1e88-155c-47e1-a46a-71ed333f2eaf None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:24.772683 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 873/1759] 10.222.0.24 () {54 vars in 912 bytes} [Fri May 27 03:35:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:35:24.780070 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6921dd7-3d56-497c-9dc2-3af93029e52d None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:24.780265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6921dd7-3d56-497c-9dc2-3af93029e52d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:24.780526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6921dd7-3d56-497c-9dc2-3af93029e52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:24.818719 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-f6921dd7-3d56-497c-9dc2-3af93029e52d None None] MFA Rules not processed for user `1af0160cd3da4a0bb391b81f083715d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:24.820213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f6921dd7-3d56-497c-9dc2-3af93029e52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:25.035531 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 887/1760] 10.222.0.24 () {58 vars in 1009 bytes} [Fri May 27 03:35:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:35:25.101896 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474d3c0-c383-4083-b121-f679d43d419e None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:25.102599 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6921dd7-3d56-497c-9dc2-3af93029e52d None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:25.103479 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474d3c0-c383-4083-b121-f679d43d419e None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:25.104072 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f6921dd7-3d56-497c-9dc2-3af93029e52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:25.104481 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474d3c0-c383-4083-b121-f679d43d419e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:25.122506 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6921dd7-3d56-497c-9dc2-3af93029e52d None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:25.128524 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501ec6d-e8e0-48f5-acb2-0861e108152e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:25.129098 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501ec6d-e8e0-48f5-acb2-0861e108152e service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:25.129378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501ec6d-e8e0-48f5-acb2-0861e108152e service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:25.129666 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501ec6d-e8e0-48f5-acb2-0861e108152e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:25.134414 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-7474d3c0-c383-4083-b121-f679d43d419e None None] MFA Rules not processed for user `1af0160cd3da4a0bb391b81f083715d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:25.137196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-7474d3c0-c383-4083-b121-f679d43d419e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:25.146652 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501ec6d-e8e0-48f5-acb2-0861e108152e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:25.148014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501ec6d-e8e0-48f5-acb2-0861e108152e service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:25.169818 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 888/1761] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:35:25.176517 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 874/1762] 10.222.0.69 () {58 vars in 1015 bytes} [Fri May 27 03:35:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:25.267956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501ec6d-e8e0-48f5-acb2-0861e108152e service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:25.332947 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501ec6d-e8e0-48f5-acb2-0861e108152e service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:25.345423 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce32cac-5380-42ad-b192-3aed4d36a676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:25.347499 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce32cac-5380-42ad-b192-3aed4d36a676 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:25.348255 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce32cac-5380-42ad-b192-3aed4d36a676 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:25.349623 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce32cac-5380-42ad-b192-3aed4d36a676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:25.360736 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce32cac-5380-42ad-b192-3aed4d36a676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:25.362508 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce32cac-5380-42ad-b192-3aed4d36a676 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:25.386115 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 889/1763] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:25.746030 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:25.907386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2215bb-89fa-44d1-8c5c-5c479ba68dfe None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:25.907871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2215bb-89fa-44d1-8c5c-5c479ba68dfe None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:25.913159 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2215bb-89fa-44d1-8c5c-5c479ba68dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:25.941448 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-1a2215bb-89fa-44d1-8c5c-5c479ba68dfe None None] MFA Rules not processed for user `dac1130950b8457ea3a9349346437357`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:25.954701 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1a2215bb-89fa-44d1-8c5c-5c479ba68dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:25.985781 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:26.107190 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 875/1764] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:26.674865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f305baa-a7fb-41c0-9d3b-45a5eaff73ea None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:26.675569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f305baa-a7fb-41c0-9d3b-45a5eaff73ea None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:26.675569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f305baa-a7fb-41c0-9d3b-45a5eaff73ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:26.774768 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-5f305baa-a7fb-41c0-9d3b-45a5eaff73ea None None] MFA Rules not processed for user `a40b8502f9634a47989c94810a34d15b`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:26.776362 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5f305baa-a7fb-41c0-9d3b-45a5eaff73ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:27.024318 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 890/1765] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:26 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:27.198620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:27.650100 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:28.508606 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f765406-9b3e-47a4-9387-3ecc21c5b2b4 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:28.509229 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f765406-9b3e-47a4-9387-3ecc21c5b2b4 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:28.509698 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f765406-9b3e-47a4-9387-3ecc21c5b2b4 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:28.511295 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 876/1766] 10.222.0.24 () {54 vars in 921 bytes} [Fri May 27 03:35:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:35:28.526768 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ebe6a-3499-48aa-bcb0-bb64e1d451de None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:28.526768 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ebe6a-3499-48aa-bcb0-bb64e1d451de None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:28.526768 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ebe6a-3499-48aa-bcb0-bb64e1d451de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:28.562042 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-bf1ebe6a-3499-48aa-bcb0-bb64e1d451de None None] MFA Rules not processed for user `95f2d0aceef4451a83ee89fac5130a0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:28.565541 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-bf1ebe6a-3499-48aa-bcb0-bb64e1d451de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:28.751725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 891/1767] 10.222.0.24 () {58 vars in 1018 bytes} [Fri May 27 03:35:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:28.833178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f765406-9b3e-47a4-9387-3ecc21c5b2b4 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:28.836205 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1f765406-9b3e-47a4-9387-3ecc21c5b2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:28.844183 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fe77e-4627-441d-8905-0d8408846cb0 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:28.844968 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fe77e-4627-441d-8905-0d8408846cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:28.845511 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697fe77e-4627-441d-8905-0d8408846cb0 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:28.850956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 892/1768] 10.222.0.24 () {54 vars in 921 bytes} [Fri May 27 03:35:28 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:35:28.875025 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e7516-5f85-4f5c-abd3-d4c27bc1be80 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:28.875535 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e7516-5f85-4f5c-abd3-d4c27bc1be80 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:28.876500 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e7516-5f85-4f5c-abd3-d4c27bc1be80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:28.906768 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:28.921842 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-f95e7516-5f85-4f5c-abd3-d4c27bc1be80 None None] MFA Rules not processed for user `22a9592143014f3287cd75c60ca17237`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:28.925648 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f95e7516-5f85-4f5c-abd3-d4c27bc1be80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:28.940062 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f765406-9b3e-47a4-9387-3ecc21c5b2b4 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:28.960515 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038fe08-40eb-4dc1-9634-710f2c63d994 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f2d0aceef4451a83ee89fac5130a0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:28.961194 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038fe08-40eb-4dc1-9634-710f2c63d994 service placement] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:28.961477 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038fe08-40eb-4dc1-9634-710f2c63d994 service placement] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:28.961743 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038fe08-40eb-4dc1-9634-710f2c63d994 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:28.977459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038fe08-40eb-4dc1-9634-710f2c63d994 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:28.986322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038fe08-40eb-4dc1-9634-710f2c63d994 service placement] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:29.005186 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 877/1769] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:29.029375 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 893/1770] 10.222.0.24 () {58 vars in 1018 bytes} [Fri May 27 03:35:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:29.045082 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038fe08-40eb-4dc1-9634-710f2c63d994 service placement] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:29.063176 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038fe08-40eb-4dc1-9634-710f2c63d994 service placement] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:29.069672 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819bb716-8eec-45ca-9e77-7d60218b3964 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:29.070404 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819bb716-8eec-45ca-9e77-7d60218b3964 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:29.070764 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819bb716-8eec-45ca-9e77-7d60218b3964 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:29.071083 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819bb716-8eec-45ca-9e77-7d60218b3964 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:29.081212 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819bb716-8eec-45ca-9e77-7d60218b3964 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:29.082814 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819bb716-8eec-45ca-9e77-7d60218b3964 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:29.093934 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 878/1771] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:29.213579 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e7516-5f85-4f5c-abd3-d4c27bc1be80 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:29.308635 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e7516-5f85-4f5c-abd3-d4c27bc1be80 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:29.315086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6f7c4-3ae3-483c-8367-ddb193ff64a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:29.316297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6f7c4-3ae3-483c-8367-ddb193ff64a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:29.316692 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6f7c4-3ae3-483c-8367-ddb193ff64a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:29.317079 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6f7c4-3ae3-483c-8367-ddb193ff64a1 admin admin] PATH_INFO: `/v3/users/8e73d92c276b4fd68bc13d784446c3fb` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:29.320040 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6f7c4-3ae3-483c-8367-ddb193ff64a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e73d92c276b4fd68bc13d784446c3fb)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:29.322270 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6f7c4-3ae3-483c-8367-ddb193ff64a1 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:29.478714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:29.566872 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-98a6f7c4-3ae3-483c-8367-ddb193ff64a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e73d92c276b4fd68bc13d784446c3fb'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:29.667176 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-98a6f7c4-3ae3-483c-8367-ddb193ff64a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e73d92c276b4fd68bc13d784446c3fb'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:29.690738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-98a6f7c4-3ae3-483c-8367-ddb193ff64a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e73d92c276b4fd68bc13d784446c3fb'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:29.691962 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-98a6f7c4-3ae3-483c-8367-ddb193ff64a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e73d92c276b4fd68bc13d784446c3fb'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:29.704114 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-98a6f7c4-3ae3-483c-8367-ddb193ff64a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e73d92c276b4fd68bc13d784446c3fb'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:29.716239 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 894/1772] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:35:29 2022] DELETE /identity/v3/users/8e73d92c276b4fd68bc13d784446c3fb => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:29.725797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7b053-4e0f-4719-8b9f-4cc1be762e41 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:29.725797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7b053-4e0f-4719-8b9f-4cc1be762e41 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:29.725797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7b053-4e0f-4719-8b9f-4cc1be762e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:29.737000 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3c33f-e5c9-4fb2-b2dc-7b4d5e697111 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:29.737289 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3c33f-e5c9-4fb2-b2dc-7b4d5e697111 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:29.737569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff3c33f-e5c9-4fb2-b2dc-7b4d5e697111 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:29.739353 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 895/1773] 10.222.0.24 () {54 vars in 921 bytes} [Fri May 27 03:35:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:35:29.746186 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-7ee7b053-4e0f-4719-8b9f-4cc1be762e41 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:29.747706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-7ee7b053-4e0f-4719-8b9f-4cc1be762e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:29.764074 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b31a02-36c1-4d4c-a933-72b3fce12330 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:29.764376 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b31a02-36c1-4d4c-a933-72b3fce12330 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:29.764654 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b31a02-36c1-4d4c-a933-72b3fce12330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:29.807697 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-c1b31a02-36c1-4d4c-a933-72b3fce12330 None None] MFA Rules not processed for user `95f2d0aceef4451a83ee89fac5130a0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:29.809042 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c1b31a02-36c1-4d4c-a933-72b3fce12330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:29.845447 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 879/1774] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:29.846467 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7b053-4e0f-4719-8b9f-4cc1be762e41 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:29.847966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-7ee7b053-4e0f-4719-8b9f-4cc1be762e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:29.900913 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 896/1775] 10.222.0.24 () {58 vars in 1018 bytes} [Fri May 27 03:35:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:29.950827 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7b053-4e0f-4719-8b9f-4cc1be762e41 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:29.956001 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21b508-e265-4280-a0d7-e5950a3e8aa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:29.956661 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21b508-e265-4280-a0d7-e5950a3e8aa1 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:29.956938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21b508-e265-4280-a0d7-e5950a3e8aa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:29.959307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e21b508-e265-4280-a0d7-e5950a3e8aa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:29.967778 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e21b508-e265-4280-a0d7-e5950a3e8aa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:29.969219 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e21b508-e265-4280-a0d7-e5950a3e8aa1 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:29.978470 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 880/1776] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:29.995945 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b31a02-36c1-4d4c-a933-72b3fce12330 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:29.997853 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c1b31a02-36c1-4d4c-a933-72b3fce12330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:30.023310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b31a02-36c1-4d4c-a933-72b3fce12330 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:30.029145 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ac7c4-c7d9-4af0-8fea-50d80941bd14 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f2d0aceef4451a83ee89fac5130a0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:30.029894 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ac7c4-c7d9-4af0-8fea-50d80941bd14 service placement] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:30.030251 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ac7c4-c7d9-4af0-8fea-50d80941bd14 service placement] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:30.030810 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ac7c4-c7d9-4af0-8fea-50d80941bd14 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:30.040245 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963ac7c4-c7d9-4af0-8fea-50d80941bd14 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:30.041911 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963ac7c4-c7d9-4af0-8fea-50d80941bd14 service placement] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:30.053372 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 897/1777] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:30.157943 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21b508-e265-4280-a0d7-e5950a3e8aa1 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:30.159525 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-5e21b508-e265-4280-a0d7-e5950a3e8aa1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:30.174687 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e21b508-e265-4280-a0d7-e5950a3e8aa1 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:30.180642 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e241b5-9877-4cd7-be1b-3f317f38af17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:30.181859 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e241b5-9877-4cd7-be1b-3f317f38af17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:30.182327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e241b5-9877-4cd7-be1b-3f317f38af17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:30.182603 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e241b5-9877-4cd7-be1b-3f317f38af17 admin admin] PATH_INFO: `/v3/users/402ec1b74c464319a5d4a5628f6f9e2a` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:30.183911 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e241b5-9877-4cd7-be1b-3f317f38af17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=402ec1b74c464319a5d4a5628f6f9e2a)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:30.185850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e241b5-9877-4cd7-be1b-3f317f38af17 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:30.401241 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-f6e241b5-9877-4cd7-be1b-3f317f38af17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '402ec1b74c464319a5d4a5628f6f9e2a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:30.463561 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-f6e241b5-9877-4cd7-be1b-3f317f38af17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '402ec1b74c464319a5d4a5628f6f9e2a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:30.464291 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-f6e241b5-9877-4cd7-be1b-3f317f38af17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '402ec1b74c464319a5d4a5628f6f9e2a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:30.472711 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-f6e241b5-9877-4cd7-be1b-3f317f38af17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '402ec1b74c464319a5d4a5628f6f9e2a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:30.473459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-f6e241b5-9877-4cd7-be1b-3f317f38af17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '402ec1b74c464319a5d4a5628f6f9e2a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:30.503935 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 881/1778] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:35:30 2022] DELETE /identity/v3/users/402ec1b74c464319a5d4a5628f6f9e2a => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:30.627807 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ac7c4-c7d9-4af0-8fea-50d80941bd14 service placement] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:30.632827 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-963ac7c4-c7d9-4af0-8fea-50d80941bd14 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:30.655608 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ac7c4-c7d9-4af0-8fea-50d80941bd14 service placement] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:30.709663 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:30.784446 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e25d0-f8e9-4c5e-a510-da85b41dfcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:30.785356 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e25d0-f8e9-4c5e-a510-da85b41dfcdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:30.785641 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e25d0-f8e9-4c5e-a510-da85b41dfcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:30.785916 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e25d0-f8e9-4c5e-a510-da85b41dfcdd admin admin] PATH_INFO: `/v3/projects/ab2b8ea6e01c4ef3ab335de16a9eb842` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:30.787882 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e25d0-f8e9-4c5e-a510-da85b41dfcdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab2b8ea6e01c4ef3ab335de16a9eb842)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:30.789840 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e25d0-f8e9-4c5e-a510-da85b41dfcdd admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:30.853173 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e241b5-9877-4cd7-be1b-3f317f38af17 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:30.855227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f6e241b5-9877-4cd7-be1b-3f317f38af17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:30.871387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e241b5-9877-4cd7-be1b-3f317f38af17 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:30.890305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f65b35-f3cb-46d6-a2ba-b99354c62c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:30.894633 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f65b35-f3cb-46d6-a2ba-b99354c62c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:30.894633 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f65b35-f3cb-46d6-a2ba-b99354c62c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:30.894633 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f65b35-f3cb-46d6-a2ba-b99354c62c39 admin admin] PATH_INFO: `/v3/projects/46545b5ceca74e959623f6e9d053da68` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:30.894853 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f65b35-f3cb-46d6-a2ba-b99354c62c39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46545b5ceca74e959623f6e9d053da68)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:30.897880 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f65b35-f3cb-46d6-a2ba-b99354c62c39 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:30.941511 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f73e25d0-f8e9-4c5e-a510-da85b41dfcdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab2b8ea6e01c4ef3ab335de16a9eb842', 'request_id': 'req-f73e25d0-f8e9-4c5e-a510-da85b41dfcdd'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:31.052470 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-96f65b35-f3cb-46d6-a2ba-b99354c62c39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46545b5ceca74e959623f6e9d053da68', 'request_id': 'req-96f65b35-f3cb-46d6-a2ba-b99354c62c39'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:31.108647 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f73e25d0-f8e9-4c5e-a510-da85b41dfcdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab2b8ea6e01c4ef3ab335de16a9eb842', 'request_id': 'req-f73e25d0-f8e9-4c5e-a510-da85b41dfcdd'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:31.117117 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f73e25d0-f8e9-4c5e-a510-da85b41dfcdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab2b8ea6e01c4ef3ab335de16a9eb842', 'request_id': 'req-f73e25d0-f8e9-4c5e-a510-da85b41dfcdd'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:31.130889 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f73e25d0-f8e9-4c5e-a510-da85b41dfcdd admin admin] The token cache is being invalidate because project ab2b8ea6e01c4ef3ab335de16a9eb842 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:35:31.130889 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f73e25d0-f8e9-4c5e-a510-da85b41dfcdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:31.132181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f73e25d0-f8e9-4c5e-a510-da85b41dfcdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:31.143536 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 898/1779] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:35:30 2022] DELETE /identity/v3/projects/ab2b8ea6e01c4ef3ab335de16a9eb842 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:31.148992 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e25d0-f8e9-4c5e-a510-da85b41dfcdd admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:31.150692 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f73e25d0-f8e9-4c5e-a510-da85b41dfcdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:31.173660 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:31.203630 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-96f65b35-f3cb-46d6-a2ba-b99354c62c39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46545b5ceca74e959623f6e9d053da68', 'request_id': 'req-96f65b35-f3cb-46d6-a2ba-b99354c62c39'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:31.205317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-96f65b35-f3cb-46d6-a2ba-b99354c62c39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46545b5ceca74e959623f6e9d053da68', 'request_id': 'req-96f65b35-f3cb-46d6-a2ba-b99354c62c39'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:31.217112 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-96f65b35-f3cb-46d6-a2ba-b99354c62c39 admin admin] The token cache is being invalidate because project 46545b5ceca74e959623f6e9d053da68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:35:31.217673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-96f65b35-f3cb-46d6-a2ba-b99354c62c39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:31.218510 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-96f65b35-f3cb-46d6-a2ba-b99354c62c39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:31.220856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 882/1780] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:35:30 2022] DELETE /identity/v3/projects/46545b5ceca74e959623f6e9d053da68 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:35:31.224621 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f65b35-f3cb-46d6-a2ba-b99354c62c39 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:31.226663 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-96f65b35-f3cb-46d6-a2ba-b99354c62c39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:31.245940 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e25d0-f8e9-4c5e-a510-da85b41dfcdd admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:31.294569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f65b35-f3cb-46d6-a2ba-b99354c62c39 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:31.300566 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208eed3d-22af-46ab-a3b0-9514f934192b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:31.301515 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208eed3d-22af-46ab-a3b0-9514f934192b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:31.304658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208eed3d-22af-46ab-a3b0-9514f934192b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:31.304658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208eed3d-22af-46ab-a3b0-9514f934192b admin admin] PATH_INFO: `/v3/users/a40b8502f9634a47989c94810a34d15b` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:31.304658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208eed3d-22af-46ab-a3b0-9514f934192b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a40b8502f9634a47989c94810a34d15b)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:31.305796 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208eed3d-22af-46ab-a3b0-9514f934192b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:31.338295 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8d48b-ddc8-48c0-ab11-095cbcad136d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:31.339439 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8d48b-ddc8-48c0-ab11-095cbcad136d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:31.339797 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8d48b-ddc8-48c0-ab11-095cbcad136d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:31.340149 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8d48b-ddc8-48c0-ab11-095cbcad136d admin admin] PATH_INFO: `/v3/users/500365dfa78b4525b292c24a2678773e` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:31.341399 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8d48b-ddc8-48c0-ab11-095cbcad136d admin admin] RBAC: Authorizing `identity:delete_user(user_id=500365dfa78b4525b292c24a2678773e)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:31.343398 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8d48b-ddc8-48c0-ab11-095cbcad136d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:31.483216 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-208eed3d-22af-46ab-a3b0-9514f934192b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a40b8502f9634a47989c94810a34d15b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:31.519394 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-51a8d48b-ddc8-48c0-ab11-095cbcad136d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '500365dfa78b4525b292c24a2678773e'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:31.589981 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-208eed3d-22af-46ab-a3b0-9514f934192b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a40b8502f9634a47989c94810a34d15b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:31.591157 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-208eed3d-22af-46ab-a3b0-9514f934192b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a40b8502f9634a47989c94810a34d15b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:31.598907 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-208eed3d-22af-46ab-a3b0-9514f934192b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a40b8502f9634a47989c94810a34d15b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:31.599650 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-208eed3d-22af-46ab-a3b0-9514f934192b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a40b8502f9634a47989c94810a34d15b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:31.617427 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 883/1781] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:35:31 2022] DELETE /identity/v3/users/a40b8502f9634a47989c94810a34d15b => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:35:31.711042 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:31.712472 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-51a8d48b-ddc8-48c0-ab11-095cbcad136d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '500365dfa78b4525b292c24a2678773e'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:31.725782 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-51a8d48b-ddc8-48c0-ab11-095cbcad136d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '500365dfa78b4525b292c24a2678773e'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:31.729537 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-51a8d48b-ddc8-48c0-ab11-095cbcad136d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '500365dfa78b4525b292c24a2678773e'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:31.738754 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-51a8d48b-ddc8-48c0-ab11-095cbcad136d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '500365dfa78b4525b292c24a2678773e'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:31.745862 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 899/1782] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:35:31 2022] DELETE /identity/v3/users/500365dfa78b4525b292c24a2678773e => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:31.923887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:31.931830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208eed3d-22af-46ab-a3b0-9514f934192b admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:31.933232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-208eed3d-22af-46ab-a3b0-9514f934192b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:32.056299 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208eed3d-22af-46ab-a3b0-9514f934192b admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:32.173837 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab79a8ea-742b-486a-8f0e-c625e1349a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:32.174661 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79a8ea-742b-486a-8f0e-c625e1349a22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:32.174985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79a8ea-742b-486a-8f0e-c625e1349a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:32.175318 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79a8ea-742b-486a-8f0e-c625e1349a22 admin admin] PATH_INFO: `/v3/projects/b8e7feb2c46c45cba6797fe2d4af3993` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:32.180080 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab79a8ea-742b-486a-8f0e-c625e1349a22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8e7feb2c46c45cba6797fe2d4af3993)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:32.182068 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab79a8ea-742b-486a-8f0e-c625e1349a22 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:32.211012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:32.366920 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ab79a8ea-742b-486a-8f0e-c625e1349a22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8e7feb2c46c45cba6797fe2d4af3993', 'request_id': 'req-ab79a8ea-742b-486a-8f0e-c625e1349a22'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:32.380598 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8d48b-ddc8-48c0-ab11-095cbcad136d admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:32.382250 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-51a8d48b-ddc8-48c0-ab11-095cbcad136d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:32.409024 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8d48b-ddc8-48c0-ab11-095cbcad136d admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:32.446202 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ab79a8ea-742b-486a-8f0e-c625e1349a22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8e7feb2c46c45cba6797fe2d4af3993', 'request_id': 'req-ab79a8ea-742b-486a-8f0e-c625e1349a22'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:32.454817 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ab79a8ea-742b-486a-8f0e-c625e1349a22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8e7feb2c46c45cba6797fe2d4af3993', 'request_id': 'req-ab79a8ea-742b-486a-8f0e-c625e1349a22'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:32.466280 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ab79a8ea-742b-486a-8f0e-c625e1349a22 admin admin] The token cache is being invalidate because project b8e7feb2c46c45cba6797fe2d4af3993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:35:32.466587 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ab79a8ea-742b-486a-8f0e-c625e1349a22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:32.467886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ab79a8ea-742b-486a-8f0e-c625e1349a22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:32.469556 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 884/1783] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:35:31 2022] DELETE /identity/v3/projects/b8e7feb2c46c45cba6797fe2d4af3993 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:32.475147 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b4859-4d1c-41e3-a4bf-ac8b8d4250d2 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:32.475420 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b4859-4d1c-41e3-a4bf-ac8b8d4250d2 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:32.475525 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbff66-ee1d-456d-a4f6-aa62e75d46cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:32.475944 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b4859-4d1c-41e3-a4bf-ac8b8d4250d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:32.476324 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbff66-ee1d-456d-a4f6-aa62e75d46cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:32.476822 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbff66-ee1d-456d-a4f6-aa62e75d46cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:32.477088 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbff66-ee1d-456d-a4f6-aa62e75d46cb admin admin] PATH_INFO: `/v3/projects/ed022ccdd12a451288d7ee5d58c569a9` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:32.478328 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbbff66-ee1d-456d-a4f6-aa62e75d46cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed022ccdd12a451288d7ee5d58c569a9)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:32.480274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbbff66-ee1d-456d-a4f6-aa62e75d46cb admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:32.497192 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-5a7b4859-4d1c-41e3-a4bf-ac8b8d4250d2 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:32.498414 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-5a7b4859-4d1c-41e3-a4bf-ac8b8d4250d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:32.507851 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 885/1784] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:32.517537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583b0c4-b4d9-4cc7-addd-1a78174a1d05 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:32.517935 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583b0c4-b4d9-4cc7-addd-1a78174a1d05 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:32.518096 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583b0c4-b4d9-4cc7-addd-1a78174a1d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:32.542827 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-3583b0c4-b4d9-4cc7-addd-1a78174a1d05 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:32.544066 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3583b0c4-b4d9-4cc7-addd-1a78174a1d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:32.552444 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 886/1785] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:32.557118 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583b0c4-b4d9-4cc7-addd-1a78174a1d05 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:32.566605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583b0c4-b4d9-4cc7-addd-1a78174a1d05 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:32.571287 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034aaf0a-84ac-419a-b6ac-43d0b072d06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:32.571947 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034aaf0a-84ac-419a-b6ac-43d0b072d06e admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:32.572230 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034aaf0a-84ac-419a-b6ac-43d0b072d06e admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:32.572507 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034aaf0a-84ac-419a-b6ac-43d0b072d06e admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:32.573235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034aaf0a-84ac-419a-b6ac-43d0b072d06e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:35:32.573545 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034aaf0a-84ac-419a-b6ac-43d0b072d06e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:32.575450 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034aaf0a-84ac-419a-b6ac-43d0b072d06e admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:32.591156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 887/1786] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:35:32.593569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034aaf0a-84ac-419a-b6ac-43d0b072d06e admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:32.606938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034aaf0a-84ac-419a-b6ac-43d0b072d06e admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:32.608311 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-9fbbff66-ee1d-456d-a4f6-aa62e75d46cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed022ccdd12a451288d7ee5d58c569a9', 'request_id': 'req-9fbbff66-ee1d-456d-a4f6-aa62e75d46cb'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:32.686240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c1daa-45a7-40d4-96fe-0ec89b386645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:32.690825 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c1daa-45a7-40d4-96fe-0ec89b386645 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:32.690825 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c1daa-45a7-40d4-96fe-0ec89b386645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:32.690825 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178c1daa-45a7-40d4-96fe-0ec89b386645 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:32.690825 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c1daa-45a7-40d4-96fe-0ec89b386645 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:32.690825 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178c1daa-45a7-40d4-96fe-0ec89b386645 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:32.721700 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-9fbbff66-ee1d-456d-a4f6-aa62e75d46cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed022ccdd12a451288d7ee5d58c569a9', 'request_id': 'req-9fbbff66-ee1d-456d-a4f6-aa62e75d46cb'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:32.732638 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-9fbbff66-ee1d-456d-a4f6-aa62e75d46cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed022ccdd12a451288d7ee5d58c569a9', 'request_id': 'req-9fbbff66-ee1d-456d-a4f6-aa62e75d46cb'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:32.751759 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-9fbbff66-ee1d-456d-a4f6-aa62e75d46cb admin admin] The token cache is being invalidate because project ed022ccdd12a451288d7ee5d58c569a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:35:32.752039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-9fbbff66-ee1d-456d-a4f6-aa62e75d46cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:32.752703 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-9fbbff66-ee1d-456d-a4f6-aa62e75d46cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:32.754385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 900/1787] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:35:32 2022] DELETE /identity/v3/projects/ed022ccdd12a451288d7ee5d58c569a9 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:32.761829 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e3ef6-1f01-4cdd-a130-f35a36253fbd None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:32.762265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e3ef6-1f01-4cdd-a130-f35a36253fbd None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:32.762666 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e3ef6-1f01-4cdd-a130-f35a36253fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:32.769042 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:32.786988 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-a66e3ef6-1f01-4cdd-a130-f35a36253fbd None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:32.786988 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a66e3ef6-1f01-4cdd-a130-f35a36253fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:32.796834 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 888/1788] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:32 2022] POST /identity/v3/projects => generated 381 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:35:32.802585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 901/1789] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:32.803627 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c1daa-45a7-40d4-96fe-0ec89b386645 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:32.804656 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799cf26-85f4-46e6-bc73-e9c6e38f0a11 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:32.805099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799cf26-85f4-46e6-bc73-e9c6e38f0a11 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:32.805378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799cf26-85f4-46e6-bc73-e9c6e38f0a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:32.805743 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-178c1daa-45a7-40d4-96fe-0ec89b386645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:32.828788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-b799cf26-85f4-46e6-bc73-e9c6e38f0a11 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:32.830269 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b799cf26-85f4-46e6-bc73-e9c6e38f0a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:32.894908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178c1daa-45a7-40d4-96fe-0ec89b386645 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:32.901357 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6a90a-f6cb-4c99-a80d-5699b93daaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:32.901735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6a90a-f6cb-4c99-a80d-5699b93daaa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:32.902404 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6a90a-f6cb-4c99-a80d-5699b93daaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:32.902520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6a90a-f6cb-4c99-a80d-5699b93daaa4 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:32.903241 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a6a90a-f6cb-4c99-a80d-5699b93daaa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:32.905372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a6a90a-f6cb-4c99-a80d-5699b93daaa4 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:32.937829 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 902/1790] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:32.944668 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799cf26-85f4-46e6-bc73-e9c6e38f0a11 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:32.954736 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799cf26-85f4-46e6-bc73-e9c6e38f0a11 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:32.960849 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf9b83-7d56-4561-9213-1a4fd2dbf824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:32.961527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf9b83-7d56-4561-9213-1a4fd2dbf824 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:32.961813 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf9b83-7d56-4561-9213-1a4fd2dbf824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:32.962095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf9b83-7d56-4561-9213-1a4fd2dbf824 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:32.962852 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf9b83-7d56-4561-9213-1a4fd2dbf824 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:35:32.963171 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf9b83-7d56-4561-9213-1a4fd2dbf824 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:32.964800 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf9b83-7d56-4561-9213-1a4fd2dbf824 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:32.978878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 903/1791] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:35:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:35:32.982560 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf9b83-7d56-4561-9213-1a4fd2dbf824 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:32.993689 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf9b83-7d56-4561-9213-1a4fd2dbf824 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:32.999815 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a6e85-fa5e-4cc1-b243-a49967b2f95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:33.003273 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a6e85-fa5e-4cc1-b243-a49967b2f95f admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:33.003273 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a6e85-fa5e-4cc1-b243-a49967b2f95f admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:33.003273 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a6e85-fa5e-4cc1-b243-a49967b2f95f admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:33.003273 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a6e85-fa5e-4cc1-b243-a49967b2f95f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:33.004570 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a6e85-fa5e-4cc1-b243-a49967b2f95f admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:33.027197 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:33.038569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 889/1792] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:32 2022] POST /identity/v3/users => generated 291 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:35:33.042679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6a90a-f6cb-4c99-a80d-5699b93daaa4 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:33.059585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6a90a-f6cb-4c99-a80d-5699b93daaa4 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:33.067143 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed290a-4802-4b22-b244-c9b30ab6d311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:33.068006 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed290a-4802-4b22-b244-c9b30ab6d311 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:33.071501 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed290a-4802-4b22-b244-c9b30ab6d311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:33.072165 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed290a-4802-4b22-b244-c9b30ab6d311 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:33.073290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed290a-4802-4b22-b244-c9b30ab6d311 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:33.075773 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed290a-4802-4b22-b244-c9b30ab6d311 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:33.087535 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 890/1793] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:33.092350 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed290a-4802-4b22-b244-c9b30ab6d311 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:33.103098 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed290a-4802-4b22-b244-c9b30ab6d311 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:33.108046 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3760a8-1ce1-4c15-b4fa-4d8119b63bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:33.108730 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3760a8-1ce1-4c15-b4fa-4d8119b63bdd admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:33.109021 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3760a8-1ce1-4c15-b4fa-4d8119b63bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:33.109409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3760a8-1ce1-4c15-b4fa-4d8119b63bdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:33.110542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3760a8-1ce1-4c15-b4fa-4d8119b63bdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:33.115611 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3760a8-1ce1-4c15-b4fa-4d8119b63bdd admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:33.125935 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 891/1794] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:35:33.127638 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 904/1795] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:32 2022] POST /identity/v3/projects => generated 373 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:35:33.128352 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3760a8-1ce1-4c15-b4fa-4d8119b63bdd admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:33.130985 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a6e85-fa5e-4cc1-b243-a49967b2f95f admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:33.149042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3760a8-1ce1-4c15-b4fa-4d8119b63bdd admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:33.271165 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a6e85-fa5e-4cc1-b243-a49967b2f95f admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:33.275358 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526064bb-879d-43a0-ac7c-d89814ee1a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:33.276206 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840ba13-c7a1-4dba-b13e-59a025f1e5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:33.276334 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526064bb-879d-43a0-ac7c-d89814ee1a7a admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:33.276660 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526064bb-879d-43a0-ac7c-d89814ee1a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:33.277025 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526064bb-879d-43a0-ac7c-d89814ee1a7a admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:33.277124 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840ba13-c7a1-4dba-b13e-59a025f1e5a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:33.277379 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840ba13-c7a1-4dba-b13e-59a025f1e5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:33.277659 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840ba13-c7a1-4dba-b13e-59a025f1e5a3 admin admin] PATH_INFO: `/v3/projects/17fa13779428430d93fd420554c9f213/users/0433dab9503b44ae9aceb6330aa86b02/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:33.279560 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526064bb-879d-43a0-ac7c-d89814ee1a7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:33.282503 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526064bb-879d-43a0-ac7c-d89814ee1a7a admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:33.326983 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d840ba13-c7a1-4dba-b13e-59a025f1e5a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17fa13779428430d93fd420554c9f213, user_id=0433dab9503b44ae9aceb6330aa86b02, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:33.330421 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d840ba13-c7a1-4dba-b13e-59a025f1e5a3 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:33.430917 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 892/1796] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:33 2022] POST /identity/v3/users => generated 287 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:33.437561 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526064bb-879d-43a0-ac7c-d89814ee1a7a admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:33.451391 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 905/1797] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:33 2022] PUT /identity/v3/projects/17fa13779428430d93fd420554c9f213/users/0433dab9503b44ae9aceb6330aa86b02/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:33.459980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0494a-a155-470f-afcf-c503b717d16f None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:33.465207 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0494a-a155-470f-afcf-c503b717d16f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:33.465207 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0494a-a155-470f-afcf-c503b717d16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:33.465619 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526064bb-879d-43a0-ac7c-d89814ee1a7a admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:33.531241 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-63c0494a-a155-470f-afcf-c503b717d16f None None] MFA Rules not processed for user `0433dab9503b44ae9aceb6330aa86b02`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:33.532607 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-63c0494a-a155-470f-afcf-c503b717d16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:33.547192 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30077e7-bdcb-4670-980e-669d6ec86040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:33.548383 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30077e7-bdcb-4670-980e-669d6ec86040 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:33.548977 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30077e7-bdcb-4670-980e-669d6ec86040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:33.549600 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30077e7-bdcb-4670-980e-669d6ec86040 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:33.553533 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30077e7-bdcb-4670-980e-669d6ec86040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:33.554275 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30077e7-bdcb-4670-980e-669d6ec86040 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:33.582449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 893/1798] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:33.586643 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30077e7-bdcb-4670-980e-669d6ec86040 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:33.607530 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30077e7-bdcb-4670-980e-669d6ec86040 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:33.612879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd652a5-21d4-4287-a0a7-4dc38648a0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:33.621780 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd652a5-21d4-4287-a0a7-4dc38648a0e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:33.621780 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd652a5-21d4-4287-a0a7-4dc38648a0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:33.621780 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd652a5-21d4-4287-a0a7-4dc38648a0e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:33.621780 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd652a5-21d4-4287-a0a7-4dc38648a0e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:33.621780 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd652a5-21d4-4287-a0a7-4dc38648a0e0 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:33.628975 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 894/1799] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:35:33.633163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd652a5-21d4-4287-a0a7-4dc38648a0e0 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:33.648807 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd652a5-21d4-4287-a0a7-4dc38648a0e0 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:33.651311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:33.656077 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a82a3-c563-4d5b-aec0-ce4732a9030f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:33.666886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a82a3-c563-4d5b-aec0-ce4732a9030f admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:33.666886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a82a3-c563-4d5b-aec0-ce4732a9030f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:33.666886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a82a3-c563-4d5b-aec0-ce4732a9030f admin admin] PATH_INFO: `/v3/projects/871210390ac4421a84321bef42e486c6/users/f63d65fd793c42098ab4ad31643f37ac/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:33.674892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 906/1800] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:35:33.675069 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a82a3-c563-4d5b-aec0-ce4732a9030f admin admin] RBAC: Authorizing `identity:create_grant(project_id=871210390ac4421a84321bef42e486c6, user_id=f63d65fd793c42098ab4ad31643f37ac, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:33.678597 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a82a3-c563-4d5b-aec0-ce4732a9030f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:33.679386 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0494a-a155-470f-afcf-c503b717d16f None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:33.692415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0494a-a155-470f-afcf-c503b717d16f None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:33.697023 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba20702-09a9-46fe-901e-224f86b70b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:33.697698 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba20702-09a9-46fe-901e-224f86b70b11 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:33.697990 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba20702-09a9-46fe-901e-224f86b70b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:33.698264 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba20702-09a9-46fe-901e-224f86b70b11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:33.699457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba20702-09a9-46fe-901e-224f86b70b11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:33.701383 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba20702-09a9-46fe-901e-224f86b70b11 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:33.750401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 895/1801] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:33 2022] PUT /identity/v3/projects/871210390ac4421a84321bef42e486c6/users/f63d65fd793c42098ab4ad31643f37ac/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:33.755415 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a506e-bbf0-4799-a938-b076d585e715 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:33.755703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a506e-bbf0-4799-a938-b076d585e715 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:33.756112 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a506e-bbf0-4799-a938-b076d585e715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:33.779239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-f80a506e-bbf0-4799-a938-b076d585e715 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:33.780448 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f80a506e-bbf0-4799-a938-b076d585e715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:33.834421 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 907/1802] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:33 2022] POST /identity/v3/projects => generated 379 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:33.835921 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba20702-09a9-46fe-901e-224f86b70b11 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:33.850001 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 896/1803] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:33.865449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a506e-bbf0-4799-a938-b076d585e715 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:33.866896 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f80a506e-bbf0-4799-a938-b076d585e715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:33.918198 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba20702-09a9-46fe-901e-224f86b70b11 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:33.923751 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c693c5-0028-48d6-a72c-d03d60eda8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:33.924580 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c693c5-0028-48d6-a72c-d03d60eda8e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:33.924965 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c693c5-0028-48d6-a72c-d03d60eda8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:33.925352 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c693c5-0028-48d6-a72c-d03d60eda8e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:33.926330 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c693c5-0028-48d6-a72c-d03d60eda8e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:33.928560 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c693c5-0028-48d6-a72c-d03d60eda8e4 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:33.942609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a506e-bbf0-4799-a938-b076d585e715 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:34.004495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8480b-3f51-43ce-9733-e4f9be72b0c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:34.007134 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8480b-3f51-43ce-9733-e4f9be72b0c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:34.007134 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8480b-3f51-43ce-9733-e4f9be72b0c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:34.007134 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8480b-3f51-43ce-9733-e4f9be72b0c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:34.019648 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce8480b-3f51-43ce-9733-e4f9be72b0c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:34.021642 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce8480b-3f51-43ce-9733-e4f9be72b0c6 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:34.033873 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 897/1804] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:34.044718 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 908/1805] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:33 2022] POST /identity/v3/users => generated 296 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:34.050366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8480b-3f51-43ce-9733-e4f9be72b0c6 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:34.066506 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8480b-3f51-43ce-9733-e4f9be72b0c6 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:34.085660 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d3279-89c4-4cde-91f0-41eb5142fc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:34.086260 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d3279-89c4-4cde-91f0-41eb5142fc8d admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:34.086545 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d3279-89c4-4cde-91f0-41eb5142fc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:34.086873 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d3279-89c4-4cde-91f0-41eb5142fc8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:34.087852 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d3279-89c4-4cde-91f0-41eb5142fc8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:34.089481 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d3279-89c4-4cde-91f0-41eb5142fc8d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:34.101847 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 898/1806] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:34.106049 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c693c5-0028-48d6-a72c-d03d60eda8e4 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:34.116772 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c693c5-0028-48d6-a72c-d03d60eda8e4 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:34.121808 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2e65c-656c-4a7d-8e86-c8e9edcaa9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:34.122703 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2e65c-656c-4a7d-8e86-c8e9edcaa9a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:34.123025 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2e65c-656c-4a7d-8e86-c8e9edcaa9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:34.123334 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2e65c-656c-4a7d-8e86-c8e9edcaa9a4 admin admin] PATH_INFO: `/v3/projects/38a9eaf2450e4de581fcc2c8be087fd8/users/1ef9345e8e5b41f59ce8492bb6dfe216/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:34.140089 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe2e65c-656c-4a7d-8e86-c8e9edcaa9a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38a9eaf2450e4de581fcc2c8be087fd8, user_id=1ef9345e8e5b41f59ce8492bb6dfe216, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:34.143226 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe2e65c-656c-4a7d-8e86-c8e9edcaa9a4 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:34.189745 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:34.246996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 909/1807] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:34 2022] PUT /identity/v3/projects/38a9eaf2450e4de581fcc2c8be087fd8/users/1ef9345e8e5b41f59ce8492bb6dfe216/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:35:34.251017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d3279-89c4-4cde-91f0-41eb5142fc8d admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:34.263230 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d3279-89c4-4cde-91f0-41eb5142fc8d admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:34.365603 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04cc22b-30cf-425e-9e05-7c6e03bec793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:34.366310 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cc22b-30cf-425e-9e05-7c6e03bec793 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:34.366592 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cc22b-30cf-425e-9e05-7c6e03bec793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:34.366886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04cc22b-30cf-425e-9e05-7c6e03bec793 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:34.367632 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04cc22b-30cf-425e-9e05-7c6e03bec793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:34.369420 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04cc22b-30cf-425e-9e05-7c6e03bec793 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:34.385699 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 899/1808] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:34.389504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2e65c-656c-4a7d-8e86-c8e9edcaa9a4 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:34.401346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2e65c-656c-4a7d-8e86-c8e9edcaa9a4 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:34.406595 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9212bc07-cebd-47bd-8f55-c3d8d909081a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:34.407584 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212bc07-cebd-47bd-8f55-c3d8d909081a admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:34.407863 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212bc07-cebd-47bd-8f55-c3d8d909081a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:34.408140 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9212bc07-cebd-47bd-8f55-c3d8d909081a admin admin] PATH_INFO: `/v3/projects/38a9eaf2450e4de581fcc2c8be087fd8/users/1ef9345e8e5b41f59ce8492bb6dfe216/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:34.410655 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9212bc07-cebd-47bd-8f55-c3d8d909081a admin admin] RBAC: Authorizing `identity:create_grant(project_id=38a9eaf2450e4de581fcc2c8be087fd8, user_id=1ef9345e8e5b41f59ce8492bb6dfe216, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:34.412845 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9212bc07-cebd-47bd-8f55-c3d8d909081a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:34.430897 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 910/1809] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:34 2022] PUT /identity/v3/projects/38a9eaf2450e4de581fcc2c8be087fd8/users/1ef9345e8e5b41f59ce8492bb6dfe216/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:34.433800 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfae4c3-a617-4445-bb01-42c29c4cb4d9 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:34.434092 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfae4c3-a617-4445-bb01-42c29c4cb4d9 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:34.434368 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfae4c3-a617-4445-bb01-42c29c4cb4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:34.461748 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-ebfae4c3-a617-4445-bb01-42c29c4cb4d9 None None] MFA Rules not processed for user `1ef9345e8e5b41f59ce8492bb6dfe216`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:34.462952 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ebfae4c3-a617-4445-bb01-42c29c4cb4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:34.509714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:34.564466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 900/1810] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:34 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:34.578644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9212bc07-cebd-47bd-8f55-c3d8d909081a admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:34.583965 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-9212bc07-cebd-47bd-8f55-c3d8d909081a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:34.768893 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9212bc07-cebd-47bd-8f55-c3d8d909081a admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:34.768893 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9832af33-ed24-44d7-8710-839f15257866 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:34.768893 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832af33-ed24-44d7-8710-839f15257866 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:34.768893 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832af33-ed24-44d7-8710-839f15257866 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:34.769596 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9832af33-ed24-44d7-8710-839f15257866 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:34.769596 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832af33-ed24-44d7-8710-839f15257866 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:34.769596 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9832af33-ed24-44d7-8710-839f15257866 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:34.786944 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 911/1811] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:34 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:35.229020 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfae4c3-a617-4445-bb01-42c29c4cb4d9 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:35.238257 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfae4c3-a617-4445-bb01-42c29c4cb4d9 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:35.242438 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd58f992-4b82-4f55-9522-37bc12931fd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:35.243847 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58f992-4b82-4f55-9522-37bc12931fd6 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:35.243847 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58f992-4b82-4f55-9522-37bc12931fd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:35.243847 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd58f992-4b82-4f55-9522-37bc12931fd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:35.298442 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd58f992-4b82-4f55-9522-37bc12931fd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:35.299906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd58f992-4b82-4f55-9522-37bc12931fd6 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:35.309695 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 901/1812] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:36.001286 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25623f-777d-42dd-99cc-65abe7dd9cb2 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:36.001889 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25623f-777d-42dd-99cc-65abe7dd9cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:36.001970 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b25623f-777d-42dd-99cc-65abe7dd9cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:36.036698 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-0b25623f-777d-42dd-99cc-65abe7dd9cb2 None None] MFA Rules not processed for user `f63d65fd793c42098ab4ad31643f37ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:36.037949 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-0b25623f-777d-42dd-99cc-65abe7dd9cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:36.150994 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 912/1813] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:36.155578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd58f992-4b82-4f55-9522-37bc12931fd6 service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:36.280239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd58f992-4b82-4f55-9522-37bc12931fd6 service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:36.285312 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4c82a-9988-4346-a73f-0b2e583754ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:36.285930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4c82a-9988-4346-a73f-0b2e583754ee admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:36.286200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4c82a-9988-4346-a73f-0b2e583754ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:36.286486 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4c82a-9988-4346-a73f-0b2e583754ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:36.287613 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e4c82a-9988-4346-a73f-0b2e583754ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:36.289239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e4c82a-9988-4346-a73f-0b2e583754ee admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:36.329945 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:36.399890 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 902/1814] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:36 2022] POST /identity/v3/projects => generated 373 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:36.404050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25623f-777d-42dd-99cc-65abe7dd9cb2 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:36.415699 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b25623f-777d-42dd-99cc-65abe7dd9cb2 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:36.476112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000911cd-2109-4987-a56a-c2ef3c8d53fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:36.477030 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000911cd-2109-4987-a56a-c2ef3c8d53fd admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:36.477771 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000911cd-2109-4987-a56a-c2ef3c8d53fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:36.477771 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000911cd-2109-4987-a56a-c2ef3c8d53fd admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:36.478567 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000911cd-2109-4987-a56a-c2ef3c8d53fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:36.481846 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000911cd-2109-4987-a56a-c2ef3c8d53fd admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:36.567878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 913/1815] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:36 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:36.571842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4c82a-9988-4346-a73f-0b2e583754ee admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:36.582138 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4c82a-9988-4346-a73f-0b2e583754ee admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:36.586688 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17277f08-6c44-4241-a5b4-9268695ef744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:36.587358 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17277f08-6c44-4241-a5b4-9268695ef744 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:36.587677 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17277f08-6c44-4241-a5b4-9268695ef744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:36.587908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17277f08-6c44-4241-a5b4-9268695ef744 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:36.588621 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17277f08-6c44-4241-a5b4-9268695ef744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:36.590203 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17277f08-6c44-4241-a5b4-9268695ef744 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:36.600938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 903/1816] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:36.605114 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000911cd-2109-4987-a56a-c2ef3c8d53fd admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:36.615830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000911cd-2109-4987-a56a-c2ef3c8d53fd admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:36.620523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a5eca-d2ef-4774-823b-479ee65397ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:36.621415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a5eca-d2ef-4774-823b-479ee65397ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:36.621740 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a5eca-d2ef-4774-823b-479ee65397ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:36.621972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a5eca-d2ef-4774-823b-479ee65397ff admin admin] PATH_INFO: `/v3/projects/c58019bd2d474edfb227d08c601d453d/users/5996958b85d842c4a340831a6107c01c/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:36.636851 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451a5eca-d2ef-4774-823b-479ee65397ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=c58019bd2d474edfb227d08c601d453d, user_id=5996958b85d842c4a340831a6107c01c, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:36.639552 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451a5eca-d2ef-4774-823b-479ee65397ff admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:36.729272 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 914/1817] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:36 2022] PUT /identity/v3/projects/c58019bd2d474edfb227d08c601d453d/users/5996958b85d842c4a340831a6107c01c/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:36.733204 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17277f08-6c44-4241-a5b4-9268695ef744 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:36.746023 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17277f08-6c44-4241-a5b4-9268695ef744 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:36.808379 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab748cf-efca-46f5-888e-1f611744eaf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:36.812416 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab748cf-efca-46f5-888e-1f611744eaf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:36.812416 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab748cf-efca-46f5-888e-1f611744eaf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:36.812416 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab748cf-efca-46f5-888e-1f611744eaf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:36.812416 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab748cf-efca-46f5-888e-1f611744eaf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:36.812416 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab748cf-efca-46f5-888e-1f611744eaf6 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:36.823502 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 904/1818] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:36.828206 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a5eca-d2ef-4774-823b-479ee65397ff admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:36.840064 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a5eca-d2ef-4774-823b-479ee65397ff admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:36.845018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee3b4d-2563-49bc-b089-854af178df74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:36.846204 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee3b4d-2563-49bc-b089-854af178df74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:36.846566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee3b4d-2563-49bc-b089-854af178df74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:36.846869 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee3b4d-2563-49bc-b089-854af178df74 admin admin] PATH_INFO: `/v3/projects/c58019bd2d474edfb227d08c601d453d/users/5996958b85d842c4a340831a6107c01c/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:36.849919 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee3b4d-2563-49bc-b089-854af178df74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c58019bd2d474edfb227d08c601d453d, user_id=5996958b85d842c4a340831a6107c01c, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:36.852735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee3b4d-2563-49bc-b089-854af178df74 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:36.864692 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 915/1819] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:36 2022] PUT /identity/v3/projects/c58019bd2d474edfb227d08c601d453d/users/5996958b85d842c4a340831a6107c01c/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:37.215604 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:37.599606 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab748cf-efca-46f5-888e-1f611744eaf6 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:37.601368 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-cab748cf-efca-46f5-888e-1f611744eaf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:37.616893 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab748cf-efca-46f5-888e-1f611744eaf6 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:37.678831 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a15b35-29fb-4035-8ef2-a26ee04206ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:37.680966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a15b35-29fb-4035-8ef2-a26ee04206ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:37.681521 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a15b35-29fb-4035-8ef2-a26ee04206ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:37.683207 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a15b35-29fb-4035-8ef2-a26ee04206ed admin admin] PATH_INFO: `/v3/users/4d70af94c5874888b86750447f2c5fbd` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:37.684762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a15b35-29fb-4035-8ef2-a26ee04206ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d70af94c5874888b86750447f2c5fbd)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:37.690191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a15b35-29fb-4035-8ef2-a26ee04206ed admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:38.088087 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-63a15b35-29fb-4035-8ef2-a26ee04206ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d70af94c5874888b86750447f2c5fbd'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:38.193842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-63a15b35-29fb-4035-8ef2-a26ee04206ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d70af94c5874888b86750447f2c5fbd'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:38.195865 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-63a15b35-29fb-4035-8ef2-a26ee04206ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d70af94c5874888b86750447f2c5fbd'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:38.220461 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-63a15b35-29fb-4035-8ef2-a26ee04206ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d70af94c5874888b86750447f2c5fbd'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:38.225787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-63a15b35-29fb-4035-8ef2-a26ee04206ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d70af94c5874888b86750447f2c5fbd'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:38.278999 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 905/1820] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:35:37 2022] DELETE /identity/v3/users/4d70af94c5874888b86750447f2c5fbd => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:38.283474 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e873e-f5c2-4a55-880b-069d3ffde4cf None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:38.283922 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e873e-f5c2-4a55-880b-069d3ffde4cf None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:38.284011 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e873e-f5c2-4a55-880b-069d3ffde4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:38.309130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-403e873e-f5c2-4a55-880b-069d3ffde4cf None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:38.310465 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-403e873e-f5c2-4a55-880b-069d3ffde4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:38.430026 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 916/1821] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:38.440222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a15b35-29fb-4035-8ef2-a26ee04206ed admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:38.442760 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-63a15b35-29fb-4035-8ef2-a26ee04206ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:38.510434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a15b35-29fb-4035-8ef2-a26ee04206ed admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:38.514609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6aa8c0-0726-438e-862a-e0c0513e3621 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:38.515343 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6aa8c0-0726-438e-862a-e0c0513e3621 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:38.515788 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6aa8c0-0726-438e-862a-e0c0513e3621 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:38.516196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6aa8c0-0726-438e-862a-e0c0513e3621 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:38.551373 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6aa8c0-0726-438e-862a-e0c0513e3621 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:38.555563 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6aa8c0-0726-438e-862a-e0c0513e3621 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:38.572576 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 906/1822] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:39.265233 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9eba8c-99d5-4fc9-8485-e9068d6deae3 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:39.265646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9eba8c-99d5-4fc9-8485-e9068d6deae3 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:39.266017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9eba8c-99d5-4fc9-8485-e9068d6deae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:39.311355 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-5c9eba8c-99d5-4fc9-8485-e9068d6deae3 None None] MFA Rules not processed for user `5996958b85d842c4a340831a6107c01c`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:39.311355 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5c9eba8c-99d5-4fc9-8485-e9068d6deae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:39.418562 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 917/1823] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:39 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:39.434340 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6aa8c0-0726-438e-862a-e0c0513e3621 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:39.435883 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-9b6aa8c0-0726-438e-862a-e0c0513e3621 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:39.439783 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9eba8c-99d5-4fc9-8485-e9068d6deae3 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:39.443556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5c9eba8c-99d5-4fc9-8485-e9068d6deae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:39.460325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6aa8c0-0726-438e-862a-e0c0513e3621 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:39.471338 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ade8ec-71ff-4b13-896e-bccd6de4bcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:39.477598 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ade8ec-71ff-4b13-896e-bccd6de4bcb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:39.477598 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ade8ec-71ff-4b13-896e-bccd6de4bcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:39.477598 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ade8ec-71ff-4b13-896e-bccd6de4bcb0 admin admin] PATH_INFO: `/v3/projects/96d122bd8bbf4048ae9918321f065a7f` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:39.477598 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ade8ec-71ff-4b13-896e-bccd6de4bcb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96d122bd8bbf4048ae9918321f065a7f)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:39.478974 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ade8ec-71ff-4b13-896e-bccd6de4bcb0 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:39.545575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9eba8c-99d5-4fc9-8485-e9068d6deae3 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:39.550790 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9f33a-9d7e-45ee-a4be-aeb4a6705cb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:39.551661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9f33a-9d7e-45ee-a4be-aeb4a6705cb7 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:39.552153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9f33a-9d7e-45ee-a4be-aeb4a6705cb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:39.553414 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9f33a-9d7e-45ee-a4be-aeb4a6705cb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:39.555459 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-79c9f33a-9d7e-45ee-a4be-aeb4a6705cb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:39.623196 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c9f33a-9d7e-45ee-a4be-aeb4a6705cb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:39.624765 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c9f33a-9d7e-45ee-a4be-aeb4a6705cb7 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:39.636742 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 918/1824] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:39.646692 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-29ade8ec-71ff-4b13-896e-bccd6de4bcb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96d122bd8bbf4048ae9918321f065a7f', 'request_id': 'req-29ade8ec-71ff-4b13-896e-bccd6de4bcb0'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:39.650963 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:39.733607 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-29ade8ec-71ff-4b13-896e-bccd6de4bcb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96d122bd8bbf4048ae9918321f065a7f', 'request_id': 'req-29ade8ec-71ff-4b13-896e-bccd6de4bcb0'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:39.742748 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-29ade8ec-71ff-4b13-896e-bccd6de4bcb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96d122bd8bbf4048ae9918321f065a7f', 'request_id': 'req-29ade8ec-71ff-4b13-896e-bccd6de4bcb0'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:39.765176 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-29ade8ec-71ff-4b13-896e-bccd6de4bcb0 admin admin] The token cache is being invalidate because project 96d122bd8bbf4048ae9918321f065a7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:35:39.767152 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-29ade8ec-71ff-4b13-896e-bccd6de4bcb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:39.769804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-29ade8ec-71ff-4b13-896e-bccd6de4bcb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:39.774545 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 907/1825] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:35:39 2022] DELETE /identity/v3/projects/96d122bd8bbf4048ae9918321f065a7f => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:39.776060 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9f33a-9d7e-45ee-a4be-aeb4a6705cb7 service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:39.804719 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9f33a-9d7e-45ee-a4be-aeb4a6705cb7 service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:39.920266 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09a643-95b2-4cff-8100-7bbb25497a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:39.921250 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09a643-95b2-4cff-8100-7bbb25497a9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:39.921604 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09a643-95b2-4cff-8100-7bbb25497a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:39.922291 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e09a643-95b2-4cff-8100-7bbb25497a9b admin admin] PATH_INFO: `/v3/users/ac7a2382334e4623a21e8568cfea95fb` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:39.923319 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09a643-95b2-4cff-8100-7bbb25497a9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac7a2382334e4623a21e8568cfea95fb)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:39.925878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e09a643-95b2-4cff-8100-7bbb25497a9b admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:41.123404 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1e09a643-95b2-4cff-8100-7bbb25497a9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac7a2382334e4623a21e8568cfea95fb'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:41.223692 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1e09a643-95b2-4cff-8100-7bbb25497a9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac7a2382334e4623a21e8568cfea95fb'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:41.244797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5621a4-bdda-40ff-bce5-3ade0f94615e None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:41.245354 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5621a4-bdda-40ff-bce5-3ade0f94615e None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:41.245563 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5621a4-bdda-40ff-bce5-3ade0f94615e None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:41.245700 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1e09a643-95b2-4cff-8100-7bbb25497a9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac7a2382334e4623a21e8568cfea95fb'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:41.246418 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1e09a643-95b2-4cff-8100-7bbb25497a9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac7a2382334e4623a21e8568cfea95fb'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:41.247899 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 908/1826] 10.222.0.24 () {54 vars in 921 bytes} [Fri May 27 03:35:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:35:41.265731 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6324b-9b55-434e-9700-04509b290436 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:41.266206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6324b-9b55-434e-9700-04509b290436 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:41.266614 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6324b-9b55-434e-9700-04509b290436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:41.293106 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-10b6324b-9b55-434e-9700-04509b290436 None None] MFA Rules not processed for user `22a9592143014f3287cd75c60ca17237`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:41.294555 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-10b6324b-9b55-434e-9700-04509b290436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:41.304570 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1e09a643-95b2-4cff-8100-7bbb25497a9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac7a2382334e4623a21e8568cfea95fb'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:41.340814 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 919/1827] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:35:39 2022] DELETE /identity/v3/users/ac7a2382334e4623a21e8568cfea95fb => generated 0 bytes in 1558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:41.399005 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 909/1828] 10.222.0.24 () {58 vars in 1018 bytes} [Fri May 27 03:35:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:41.527484 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09a643-95b2-4cff-8100-7bbb25497a9b admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:41.528723 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1e09a643-95b2-4cff-8100-7bbb25497a9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:41.637499 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e09a643-95b2-4cff-8100-7bbb25497a9b admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:41.643185 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ae515-860d-4c33-ba01-48c97b47cb41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:41.643832 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ae515-860d-4c33-ba01-48c97b47cb41 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:41.644112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ae515-860d-4c33-ba01-48c97b47cb41 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:41.644405 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ae515-860d-4c33-ba01-48c97b47cb41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:41.655697 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ae515-860d-4c33-ba01-48c97b47cb41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:41.657588 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ae515-860d-4c33-ba01-48c97b47cb41 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:41.670222 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 920/1829] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:41.983207 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6324b-9b55-434e-9700-04509b290436 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:41.984645 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-10b6324b-9b55-434e-9700-04509b290436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:41.999067 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6324b-9b55-434e-9700-04509b290436 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:42.041481 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ae515-860d-4c33-ba01-48c97b47cb41 service nova] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:42.043550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1c9ae515-860d-4c33-ba01-48c97b47cb41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:42.058680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e44195-4367-4644-8f59-7d7a29158130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:42.059655 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e44195-4367-4644-8f59-7d7a29158130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:42.059934 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e44195-4367-4644-8f59-7d7a29158130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:42.060208 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e44195-4367-4644-8f59-7d7a29158130 admin admin] PATH_INFO: `/v3/projects/d16cb313c2d84c679864a909c3b3984e` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:42.061397 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e44195-4367-4644-8f59-7d7a29158130 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d16cb313c2d84c679864a909c3b3984e)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:42.063358 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e44195-4367-4644-8f59-7d7a29158130 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:42.102197 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:42.120816 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ae515-860d-4c33-ba01-48c97b47cb41 service nova] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:42.125392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bff627-e377-4fa7-995a-d2f1f5dd6999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:42.126267 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bff627-e377-4fa7-995a-d2f1f5dd6999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:42.126554 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bff627-e377-4fa7-995a-d2f1f5dd6999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:42.126849 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bff627-e377-4fa7-995a-d2f1f5dd6999 admin admin] PATH_INFO: `/v3/users/ed8ef1333d484db284ffb6e9498bcaec` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:42.128090 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bff627-e377-4fa7-995a-d2f1f5dd6999 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed8ef1333d484db284ffb6e9498bcaec)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:42.130037 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bff627-e377-4fa7-995a-d2f1f5dd6999 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:42.181769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c8e44195-4367-4644-8f59-7d7a29158130 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd16cb313c2d84c679864a909c3b3984e', 'request_id': 'req-c8e44195-4367-4644-8f59-7d7a29158130'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:42.301140 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c8e44195-4367-4644-8f59-7d7a29158130 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd16cb313c2d84c679864a909c3b3984e', 'request_id': 'req-c8e44195-4367-4644-8f59-7d7a29158130'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:42.308649 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c8e44195-4367-4644-8f59-7d7a29158130 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd16cb313c2d84c679864a909c3b3984e', 'request_id': 'req-c8e44195-4367-4644-8f59-7d7a29158130'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:42.319827 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c8e44195-4367-4644-8f59-7d7a29158130 admin admin] The token cache is being invalidate because project d16cb313c2d84c679864a909c3b3984e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:35:42.320116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c8e44195-4367-4644-8f59-7d7a29158130 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:42.320768 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c8e44195-4367-4644-8f59-7d7a29158130 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:42.322443 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 910/1830] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:35:41 2022] DELETE /identity/v3/projects/d16cb313c2d84c679864a909c3b3984e => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:42.327429 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b17ff-d626-4f6b-bb40-1adf97cd77fd None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:42.327716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b17ff-d626-4f6b-bb40-1adf97cd77fd None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:42.329318 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b17ff-d626-4f6b-bb40-1adf97cd77fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:42.348270 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-6d3b17ff-d626-4f6b-bb40-1adf97cd77fd None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:42.349486 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6d3b17ff-d626-4f6b-bb40-1adf97cd77fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:42.360503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-26bff627-e377-4fa7-995a-d2f1f5dd6999 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed8ef1333d484db284ffb6e9498bcaec'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:42.407468 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 911/1831] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:35:42.416224 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67878db2-d8f7-4f8d-becb-0166a149058e None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:42.416541 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67878db2-d8f7-4f8d-becb-0166a149058e None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:42.416814 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67878db2-d8f7-4f8d-becb-0166a149058e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:42.434405 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-67878db2-d8f7-4f8d-becb-0166a149058e None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:42.435625 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-67878db2-d8f7-4f8d-becb-0166a149058e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:42.443896 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 912/1832] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:35:42.447821 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67878db2-d8f7-4f8d-becb-0166a149058e None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:42.456706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67878db2-d8f7-4f8d-becb-0166a149058e None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:42.461341 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aab164-94dc-433e-97c2-91d5d8a541c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:42.462093 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aab164-94dc-433e-97c2-91d5d8a541c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:42.462385 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aab164-94dc-433e-97c2-91d5d8a541c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:42.462657 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aab164-94dc-433e-97c2-91d5d8a541c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:42.463414 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aab164-94dc-433e-97c2-91d5d8a541c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:35:42.463718 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aab164-94dc-433e-97c2-91d5d8a541c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:42.465331 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aab164-94dc-433e-97c2-91d5d8a541c1 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:42.477450 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 913/1833] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:35:42.480926 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-26bff627-e377-4fa7-995a-d2f1f5dd6999 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed8ef1333d484db284ffb6e9498bcaec'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:42.481953 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aab164-94dc-433e-97c2-91d5d8a541c1 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:42.492541 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aab164-94dc-433e-97c2-91d5d8a541c1 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:42.495311 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-26bff627-e377-4fa7-995a-d2f1f5dd6999 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed8ef1333d484db284ffb6e9498bcaec'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:42.496007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-26bff627-e377-4fa7-995a-d2f1f5dd6999 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed8ef1333d484db284ffb6e9498bcaec'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:42.497293 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec16df-5d71-4fbe-8453-247ec1667d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:42.498716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec16df-5d71-4fbe-8453-247ec1667d4a admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:42.499032 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec16df-5d71-4fbe-8453-247ec1667d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:42.499305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec16df-5d71-4fbe-8453-247ec1667d4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:42.499999 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec16df-5d71-4fbe-8453-247ec1667d4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:42.501936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec16df-5d71-4fbe-8453-247ec1667d4a admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:42.503555 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-26bff627-e377-4fa7-995a-d2f1f5dd6999 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed8ef1333d484db284ffb6e9498bcaec'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:42.509715 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 921/1834] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:35:42 2022] DELETE /identity/v3/users/ed8ef1333d484db284ffb6e9498bcaec => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:42.514481 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d08514-0293-4da3-ba03-ab3f9ee4e9f7 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:42.519271 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d08514-0293-4da3-ba03-ab3f9ee4e9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:42.519271 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d08514-0293-4da3-ba03-ab3f9ee4e9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:42.533567 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-47d08514-0293-4da3-ba03-ab3f9ee4e9f7 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:42.534767 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-47d08514-0293-4da3-ba03-ab3f9ee4e9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:42.598066 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 922/1835] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:42.610501 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d08514-0293-4da3-ba03-ab3f9ee4e9f7 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:42.617548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 914/1836] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:42 2022] POST /identity/v3/projects => generated 381 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:35:42.627338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-47d08514-0293-4da3-ba03-ab3f9ee4e9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:42.628467 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec16df-5d71-4fbe-8453-247ec1667d4a admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:42.630116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-16ec16df-5d71-4fbe-8453-247ec1667d4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:42.701200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d08514-0293-4da3-ba03-ab3f9ee4e9f7 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:42.730183 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec16df-5d71-4fbe-8453-247ec1667d4a admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:42.736533 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8bf69-9aef-402d-863c-7415e65b66bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:42.737636 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8bf69-9aef-402d-863c-7415e65b66bb admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:42.738378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8bf69-9aef-402d-863c-7415e65b66bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:42.738776 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8bf69-9aef-402d-863c-7415e65b66bb admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:42.740223 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b8bf69-9aef-402d-863c-7415e65b66bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:42.747258 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b8bf69-9aef-402d-863c-7415e65b66bb admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:42.790655 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954da327-3021-4d55-9970-78953f7376bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:42.791686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954da327-3021-4d55-9970-78953f7376bc service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:42.792037 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954da327-3021-4d55-9970-78953f7376bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:42.792391 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954da327-3021-4d55-9970-78953f7376bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:42.803294 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954da327-3021-4d55-9970-78953f7376bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:42.804755 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954da327-3021-4d55-9970-78953f7376bc service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:42.815533 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 923/1837] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:42.824765 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 915/1838] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:42 2022] POST /identity/v3/users => generated 291 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:42.828832 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954da327-3021-4d55-9970-78953f7376bc service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:42.839414 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954da327-3021-4d55-9970-78953f7376bc service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:42.844396 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98588d34-f480-44f8-aadf-efde596a78c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:42.845606 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98588d34-f480-44f8-aadf-efde596a78c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:42.845893 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98588d34-f480-44f8-aadf-efde596a78c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:42.850537 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98588d34-f480-44f8-aadf-efde596a78c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:42.850537 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98588d34-f480-44f8-aadf-efde596a78c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:42.850537 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98588d34-f480-44f8-aadf-efde596a78c7 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:42.876751 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 924/1839] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:42.880224 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8bf69-9aef-402d-863c-7415e65b66bb admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:42.891173 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8bf69-9aef-402d-863c-7415e65b66bb admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:42.898870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df434ba3-a5ef-443a-ae1f-6cd9f761d618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:42.899567 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df434ba3-a5ef-443a-ae1f-6cd9f761d618 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:42.900035 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df434ba3-a5ef-443a-ae1f-6cd9f761d618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:42.900322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df434ba3-a5ef-443a-ae1f-6cd9f761d618 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:42.901255 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df434ba3-a5ef-443a-ae1f-6cd9f761d618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:42.902743 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df434ba3-a5ef-443a-ae1f-6cd9f761d618 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:42.926750 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 916/1840] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:42.928464 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98588d34-f480-44f8-aadf-efde596a78c7 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:42.940399 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98588d34-f480-44f8-aadf-efde596a78c7 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:42.954098 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd928bd-53d8-4329-b2e0-fc87cb45973d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:42.959486 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd928bd-53d8-4329-b2e0-fc87cb45973d admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:42.959671 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd928bd-53d8-4329-b2e0-fc87cb45973d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:42.959925 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd928bd-53d8-4329-b2e0-fc87cb45973d admin admin] PATH_INFO: `/v3/projects/4b070c4749c34407b659daf91ea6a30a/users/8e60097a8ab04f55866b2190f409cd29/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:42.979956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd928bd-53d8-4329-b2e0-fc87cb45973d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b070c4749c34407b659daf91ea6a30a, user_id=8e60097a8ab04f55866b2190f409cd29, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:42.983538 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd928bd-53d8-4329-b2e0-fc87cb45973d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:43.086693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 925/1841] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:42 2022] PUT /identity/v3/projects/4b070c4749c34407b659daf91ea6a30a/users/8e60097a8ab04f55866b2190f409cd29/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:35:43.097538 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752731d-120f-445d-a399-4f798997785c None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:43.097929 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752731d-120f-445d-a399-4f798997785c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:43.098285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752731d-120f-445d-a399-4f798997785c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:43.129509 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-c752731d-120f-445d-a399-4f798997785c None None] MFA Rules not processed for user `8e60097a8ab04f55866b2190f409cd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:43.130825 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c752731d-120f-445d-a399-4f798997785c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:43.212329 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 917/1842] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:43.217596 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd928bd-53d8-4329-b2e0-fc87cb45973d admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:43.312970 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd928bd-53d8-4329-b2e0-fc87cb45973d admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:43.319751 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee486a18-bfae-402e-848d-5a85e572b1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:43.325794 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee486a18-bfae-402e-848d-5a85e572b1b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:43.325794 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee486a18-bfae-402e-848d-5a85e572b1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:43.325794 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee486a18-bfae-402e-848d-5a85e572b1b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:43.325794 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee486a18-bfae-402e-848d-5a85e572b1b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:43.325794 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee486a18-bfae-402e-848d-5a85e572b1b6 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:43.418789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 926/1843] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:43 2022] POST /identity/v3/projects => generated 377 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:43.421586 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752731d-120f-445d-a399-4f798997785c None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:43.433967 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752731d-120f-445d-a399-4f798997785c None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:43.510758 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcf97f-a451-4653-8e35-6731243bbe38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:43.511478 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcf97f-a451-4653-8e35-6731243bbe38 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:43.511764 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcf97f-a451-4653-8e35-6731243bbe38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:43.512039 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcf97f-a451-4653-8e35-6731243bbe38 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:43.513107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bcf97f-a451-4653-8e35-6731243bbe38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:43.516519 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bcf97f-a451-4653-8e35-6731243bbe38 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:43.603965 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee486a18-bfae-402e-848d-5a85e572b1b6 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:43.605851 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ee486a18-bfae-402e-848d-5a85e572b1b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:43.631499 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee486a18-bfae-402e-848d-5a85e572b1b6 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:43.635258 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 918/1844] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:43 2022] POST /identity/v3/users => generated 295 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:35:43.637049 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd7263-cbb5-466e-b201-cb106b8e10bc None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:43.637342 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd7263-cbb5-466e-b201-cb106b8e10bc None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:43.638038 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acd7263-cbb5-466e-b201-cb106b8e10bc None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:43.639698 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 919/1845] 10.222.0.24 () {54 vars in 921 bytes} [Fri May 27 03:35:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:35:43.640886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd7263-cbb5-466e-b201-cb106b8e10bc None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:43.641695 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad5422-b98f-466c-a72b-28eb717659bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:43.642782 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad5422-b98f-466c-a72b-28eb717659bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:43.643276 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad5422-b98f-466c-a72b-28eb717659bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:43.643760 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ad5422-b98f-466c-a72b-28eb717659bf admin admin] PATH_INFO: `/v3/projects/440a5550bef3447eaae6b5f1a3674116` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:43.645824 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad5422-b98f-466c-a72b-28eb717659bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=440a5550bef3447eaae6b5f1a3674116)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:43.648381 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ad5422-b98f-466c-a72b-28eb717659bf admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:43.658502 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acd7263-cbb5-466e-b201-cb106b8e10bc None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:43.665725 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d204766-9020-4b85-854e-155525bb8826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:43.666516 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d204766-9020-4b85-854e-155525bb8826 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:43.666802 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d204766-9020-4b85-854e-155525bb8826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:43.667234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d204766-9020-4b85-854e-155525bb8826 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:43.669280 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d204766-9020-4b85-854e-155525bb8826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:43.671169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d204766-9020-4b85-854e-155525bb8826 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:43.701128 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 920/1846] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:43.701128 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9bfa6-4d51-48b7-90cf-017c5988c89a None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:43.701128 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9bfa6-4d51-48b7-90cf-017c5988c89a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:43.701128 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9bfa6-4d51-48b7-90cf-017c5988c89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:43.732052 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-fad9bfa6-4d51-48b7-90cf-017c5988c89a None None] MFA Rules not processed for user `22a9592143014f3287cd75c60ca17237`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:43.733489 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-fad9bfa6-4d51-48b7-90cf-017c5988c89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:43.846677 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 921/1847] 10.222.0.24 () {58 vars in 1018 bytes} [Fri May 27 03:35:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:43.848251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9bfa6-4d51-48b7-90cf-017c5988c89a None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:43.856423 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f3ad5422-b98f-466c-a72b-28eb717659bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '440a5550bef3447eaae6b5f1a3674116', 'request_id': 'req-f3ad5422-b98f-466c-a72b-28eb717659bf'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:43.873337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9bfa6-4d51-48b7-90cf-017c5988c89a None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:43.968793 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc09e1-2b55-4dc1-952e-9260dc07fee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:43.969719 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc09e1-2b55-4dc1-952e-9260dc07fee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:43.970004 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc09e1-2b55-4dc1-952e-9260dc07fee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:43.970290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc09e1-2b55-4dc1-952e-9260dc07fee6 admin admin] PATH_INFO: `/v3/projects/63088692fa9540dd843c2c50ad945d8c/users/d64b3bf418ae40cda178f693197a05d4/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:43.981979 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f3ad5422-b98f-466c-a72b-28eb717659bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '440a5550bef3447eaae6b5f1a3674116', 'request_id': 'req-f3ad5422-b98f-466c-a72b-28eb717659bf'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:43.994215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f3ad5422-b98f-466c-a72b-28eb717659bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '440a5550bef3447eaae6b5f1a3674116', 'request_id': 'req-f3ad5422-b98f-466c-a72b-28eb717659bf'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:44.001412 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dc09e1-2b55-4dc1-952e-9260dc07fee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63088692fa9540dd843c2c50ad945d8c, user_id=d64b3bf418ae40cda178f693197a05d4, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:44.004876 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dc09e1-2b55-4dc1-952e-9260dc07fee6 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:44.008862 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f3ad5422-b98f-466c-a72b-28eb717659bf admin admin] The token cache is being invalidate because project 440a5550bef3447eaae6b5f1a3674116 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:35:44.009270 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f3ad5422-b98f-466c-a72b-28eb717659bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:44.010054 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-f3ad5422-b98f-466c-a72b-28eb717659bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:44.015158 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 927/1848] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:35:43 2022] DELETE /identity/v3/projects/440a5550bef3447eaae6b5f1a3674116 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:44.016188 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad5422-b98f-466c-a72b-28eb717659bf admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:44.017908 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f3ad5422-b98f-466c-a72b-28eb717659bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:44.092438 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 922/1849] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:43 2022] PUT /identity/v3/projects/63088692fa9540dd843c2c50ad945d8c/users/d64b3bf418ae40cda178f693197a05d4/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:44.095284 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc09e1-2b55-4dc1-952e-9260dc07fee6 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:44.095396 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ad5422-b98f-466c-a72b-28eb717659bf admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:44.096207 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-09dc09e1-2b55-4dc1-952e-9260dc07fee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:44.168721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85862635-17c1-4f55-8e24-ce2ac29f514d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:44.169427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85862635-17c1-4f55-8e24-ce2ac29f514d service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:44.169712 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85862635-17c1-4f55-8e24-ce2ac29f514d service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:44.169969 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85862635-17c1-4f55-8e24-ce2ac29f514d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:44.172137 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-85862635-17c1-4f55-8e24-ce2ac29f514d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:44.235719 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc09e1-2b55-4dc1-952e-9260dc07fee6 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:44.244827 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755905d-ac4f-4492-995f-b9d1730feb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:44.246289 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755905d-ac4f-4492-995f-b9d1730feb20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:44.246573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755905d-ac4f-4492-995f-b9d1730feb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:44.246861 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755905d-ac4f-4492-995f-b9d1730feb20 admin admin] PATH_INFO: `/v3/users/e600d24c9e0442538f457dcccd52f581` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:44.248442 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6755905d-ac4f-4492-995f-b9d1730feb20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e600d24c9e0442538f457dcccd52f581)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:44.250851 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6755905d-ac4f-4492-995f-b9d1730feb20 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:44.260084 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85862635-17c1-4f55-8e24-ce2ac29f514d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:44.261679 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85862635-17c1-4f55-8e24-ce2ac29f514d service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:44.274123 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 928/1850] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:44 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:44.275252 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85862635-17c1-4f55-8e24-ce2ac29f514d service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:44.277815 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-85862635-17c1-4f55-8e24-ce2ac29f514d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:44.294775 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85862635-17c1-4f55-8e24-ce2ac29f514d service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:44.300304 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7cd16d-eb2a-4c6c-a271-622b10eec201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:44.301258 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7cd16d-eb2a-4c6c-a271-622b10eec201 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:44.301622 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7cd16d-eb2a-4c6c-a271-622b10eec201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:44.301972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7cd16d-eb2a-4c6c-a271-622b10eec201 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:44.303342 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7cd16d-eb2a-4c6c-a271-622b10eec201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:44.305202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7cd16d-eb2a-4c6c-a271-622b10eec201 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:44.316615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 929/1851] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:44.319844 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7cd16d-eb2a-4c6c-a271-622b10eec201 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:44.327667 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-0a7cd16d-eb2a-4c6c-a271-622b10eec201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:44.415344 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7cd16d-eb2a-4c6c-a271-622b10eec201 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:44.421816 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083377c-b70c-44e4-9950-c6316d1345cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:44.423276 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083377c-b70c-44e4-9950-c6316d1345cf service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:44.424163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083377c-b70c-44e4-9950-c6316d1345cf service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:44.424437 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083377c-b70c-44e4-9950-c6316d1345cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:44.426514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e083377c-b70c-44e4-9950-c6316d1345cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:44.440381 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083377c-b70c-44e4-9950-c6316d1345cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:44.442360 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083377c-b70c-44e4-9950-c6316d1345cf service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:44.454619 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 930/1852] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:44.455530 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083377c-b70c-44e4-9950-c6316d1345cf service nova] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:44.466282 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083377c-b70c-44e4-9950-c6316d1345cf service nova] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:44.471374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fac599-e00a-466a-bf23-644c8c762214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:44.472237 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fac599-e00a-466a-bf23-644c8c762214 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:44.473154 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fac599-e00a-466a-bf23-644c8c762214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:44.473154 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fac599-e00a-466a-bf23-644c8c762214 admin admin] PATH_INFO: `/v3/projects/63088692fa9540dd843c2c50ad945d8c/users/d64b3bf418ae40cda178f693197a05d4/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:44.475128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fac599-e00a-466a-bf23-644c8c762214 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63088692fa9540dd843c2c50ad945d8c, user_id=d64b3bf418ae40cda178f693197a05d4, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:44.477973 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fac599-e00a-466a-bf23-644c8c762214 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:44.496963 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 931/1853] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:44 2022] PUT /identity/v3/projects/63088692fa9540dd843c2c50ad945d8c/users/d64b3bf418ae40cda178f693197a05d4/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:44.508209 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e06439-3901-4e91-97e3-46b3914f15de None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:44.508564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e06439-3901-4e91-97e3-46b3914f15de None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:44.508847 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e06439-3901-4e91-97e3-46b3914f15de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:44.519178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:44.544906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6755905d-ac4f-4492-995f-b9d1730feb20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e600d24c9e0442538f457dcccd52f581'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:44.571894 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-a8e06439-3901-4e91-97e3-46b3914f15de None None] MFA Rules not processed for user `d64b3bf418ae40cda178f693197a05d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:44.572795 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a8e06439-3901-4e91-97e3-46b3914f15de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:44.631515 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6755905d-ac4f-4492-995f-b9d1730feb20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e600d24c9e0442538f457dcccd52f581'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:44.632157 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6755905d-ac4f-4492-995f-b9d1730feb20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e600d24c9e0442538f457dcccd52f581'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:44.645871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6755905d-ac4f-4492-995f-b9d1730feb20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e600d24c9e0442538f457dcccd52f581'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:44.646609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6755905d-ac4f-4492-995f-b9d1730feb20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e600d24c9e0442538f457dcccd52f581'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:44.680355 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 932/1854] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:44 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:44.686688 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 923/1855] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:35:44 2022] DELETE /identity/v3/users/e600d24c9e0442538f457dcccd52f581 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:44.775600 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e06439-3901-4e91-97e3-46b3914f15de None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:44.777403 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a8e06439-3901-4e91-97e3-46b3914f15de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:44.869915 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e06439-3901-4e91-97e3-46b3914f15de None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:44.957217 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312afcf-0e60-4200-aff4-cffc2478d320 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:44.957217 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312afcf-0e60-4200-aff4-cffc2478d320 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:44.957217 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312afcf-0e60-4200-aff4-cffc2478d320 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:44.957217 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312afcf-0e60-4200-aff4-cffc2478d320 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:45.032974 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6312afcf-0e60-4200-aff4-cffc2478d320 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:45.034419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6312afcf-0e60-4200-aff4-cffc2478d320 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:45.049692 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 933/1856] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:44 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:45.288153 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755905d-ac4f-4492-995f-b9d1730feb20 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:45.290213 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6755905d-ac4f-4492-995f-b9d1730feb20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:45.386900 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755905d-ac4f-4492-995f-b9d1730feb20 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:45.391859 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5acdf4-f101-4203-be9a-38e65dcceef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:45.392732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5acdf4-f101-4203-be9a-38e65dcceef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:45.393130 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5acdf4-f101-4203-be9a-38e65dcceef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:45.393333 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5acdf4-f101-4203-be9a-38e65dcceef9 admin admin] PATH_INFO: `/v3/projects/e007b1b172b64559be6ada4d6b1a4f33` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:45.394517 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5acdf4-f101-4203-be9a-38e65dcceef9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e007b1b172b64559be6ada4d6b1a4f33)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:45.399110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5acdf4-f101-4203-be9a-38e65dcceef9 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:45.542547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-be5acdf4-f101-4203-be9a-38e65dcceef9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e007b1b172b64559be6ada4d6b1a4f33', 'request_id': 'req-be5acdf4-f101-4203-be9a-38e65dcceef9'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:45.546206 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:45.658572 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-be5acdf4-f101-4203-be9a-38e65dcceef9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e007b1b172b64559be6ada4d6b1a4f33', 'request_id': 'req-be5acdf4-f101-4203-be9a-38e65dcceef9'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:45.666606 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-be5acdf4-f101-4203-be9a-38e65dcceef9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e007b1b172b64559be6ada4d6b1a4f33', 'request_id': 'req-be5acdf4-f101-4203-be9a-38e65dcceef9'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:45.678733 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-be5acdf4-f101-4203-be9a-38e65dcceef9 admin admin] The token cache is being invalidate because project e007b1b172b64559be6ada4d6b1a4f33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:35:45.679096 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-be5acdf4-f101-4203-be9a-38e65dcceef9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:45.679902 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-be5acdf4-f101-4203-be9a-38e65dcceef9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:45.681495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 924/1857] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:35:45 2022] DELETE /identity/v3/projects/e007b1b172b64559be6ada4d6b1a4f33 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:45.686710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2dd54-71c3-4813-8638-a0d7dc2cd840 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:45.686998 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2dd54-71c3-4813-8638-a0d7dc2cd840 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:45.687278 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2dd54-71c3-4813-8638-a0d7dc2cd840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:45.713709 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-82a2dd54-71c3-4813-8638-a0d7dc2cd840 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:45.714987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-82a2dd54-71c3-4813-8638-a0d7dc2cd840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:45.776276 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 934/1858] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:45.784202 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b06213-f5c4-40ec-89d6-8caf56d42d10 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:45.784496 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b06213-f5c4-40ec-89d6-8caf56d42d10 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:45.784784 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b06213-f5c4-40ec-89d6-8caf56d42d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:45.819068 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2dd54-71c3-4813-8638-a0d7dc2cd840 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:45.821423 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-82a2dd54-71c3-4813-8638-a0d7dc2cd840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:45.825494 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d5b06213-f5c4-40ec-89d6-8caf56d42d10 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:45.827035 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d5b06213-f5c4-40ec-89d6-8caf56d42d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:45.837579 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 925/1859] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:45.841991 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b06213-f5c4-40ec-89d6-8caf56d42d10 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:45.842266 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a2dd54-71c3-4813-8638-a0d7dc2cd840 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:45.849781 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b2126-9c86-4d06-930e-8e4b477d4cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:45.852623 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b2126-9c86-4d06-930e-8e4b477d4cf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:45.853648 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b2126-9c86-4d06-930e-8e4b477d4cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:45.854086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b2126-9c86-4d06-930e-8e4b477d4cf3 admin admin] PATH_INFO: `/v3/users/8e60097a8ab04f55866b2190f409cd29` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:45.855989 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b2126-9c86-4d06-930e-8e4b477d4cf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e60097a8ab04f55866b2190f409cd29)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:45.856501 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b06213-f5c4-40ec-89d6-8caf56d42d10 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:45.858444 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b2126-9c86-4d06-930e-8e4b477d4cf3 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:45.862713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e555b-84c6-46da-9205-40e0d986ba4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:45.863806 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e555b-84c6-46da-9205-40e0d986ba4c admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:45.865200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e555b-84c6-46da-9205-40e0d986ba4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:45.865637 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e555b-84c6-46da-9205-40e0d986ba4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:45.866751 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e555b-84c6-46da-9205-40e0d986ba4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:35:45.868062 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e555b-84c6-46da-9205-40e0d986ba4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:45.872193 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e555b-84c6-46da-9205-40e0d986ba4c admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:45.896726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 926/1860] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:35:45.897805 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e555b-84c6-46da-9205-40e0d986ba4c admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:45.921372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e555b-84c6-46da-9205-40e0d986ba4c admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:45.929293 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76d011-026e-46cc-b528-316bee7c7b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:45.930065 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76d011-026e-46cc-b528-316bee7c7b55 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:45.930353 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76d011-026e-46cc-b528-316bee7c7b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:45.930806 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76d011-026e-46cc-b528-316bee7c7b55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:45.931579 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76d011-026e-46cc-b528-316bee7c7b55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:45.933879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76d011-026e-46cc-b528-316bee7c7b55 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:46.044696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 927/1861] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:45 2022] POST /identity/v3/projects => generated 377 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:46.045837 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76d011-026e-46cc-b528-316bee7c7b55 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:46.118805 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a73b2126-9c86-4d06-930e-8e4b477d4cf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e60097a8ab04f55866b2190f409cd29'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:46.133091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76d011-026e-46cc-b528-316bee7c7b55 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:46.137835 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ca85b-539d-4097-be15-6d9322d0a4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:46.138590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ca85b-539d-4097-be15-6d9322d0a4a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:46.138983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ca85b-539d-4097-be15-6d9322d0a4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:46.139329 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ca85b-539d-4097-be15-6d9322d0a4a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:46.140091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969ca85b-539d-4097-be15-6d9322d0a4a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:46.142134 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969ca85b-539d-4097-be15-6d9322d0a4a0 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:46.301493 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a73b2126-9c86-4d06-930e-8e4b477d4cf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e60097a8ab04f55866b2190f409cd29'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:46.313561 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 928/1862] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:46 2022] POST /identity/v3/users => generated 289 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:46.317633 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ca85b-539d-4097-be15-6d9322d0a4a0 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:46.321879 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a73b2126-9c86-4d06-930e-8e4b477d4cf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e60097a8ab04f55866b2190f409cd29'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:46.322892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a73b2126-9c86-4d06-930e-8e4b477d4cf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e60097a8ab04f55866b2190f409cd29'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:46.333347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a73b2126-9c86-4d06-930e-8e4b477d4cf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e60097a8ab04f55866b2190f409cd29'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:46.336337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ca85b-539d-4097-be15-6d9322d0a4a0 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:46.341529 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 935/1863] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:35:45 2022] DELETE /identity/v3/users/8e60097a8ab04f55866b2190f409cd29 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:46.343854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b8897-a001-4de6-9f1d-ba07b5e01ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:46.344997 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b8897-a001-4de6-9f1d-ba07b5e01ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:46.345585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b8897-a001-4de6-9f1d-ba07b5e01ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:46.346499 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25790669-7dd8-4b5a-8e03-3ebf7021107c None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:46.346628 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b8897-a001-4de6-9f1d-ba07b5e01ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:46.347112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25790669-7dd8-4b5a-8e03-3ebf7021107c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:46.347534 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b8897-a001-4de6-9f1d-ba07b5e01ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:46.347642 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25790669-7dd8-4b5a-8e03-3ebf7021107c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:46.350606 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b8897-a001-4de6-9f1d-ba07b5e01ea7 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:46.362247 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 929/1864] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:35:46.366594 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b8897-a001-4de6-9f1d-ba07b5e01ea7 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:46.368544 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b51b8897-a001-4de6-9f1d-ba07b5e01ea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:46.369207 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-25790669-7dd8-4b5a-8e03-3ebf7021107c None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:46.370510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-25790669-7dd8-4b5a-8e03-3ebf7021107c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:46.382081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b8897-a001-4de6-9f1d-ba07b5e01ea7 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:46.435923 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 936/1865] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:35:46.443199 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0d785-89c3-485d-b298-9514bc3c953b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:46.454498 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0d785-89c3-485d-b298-9514bc3c953b admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:46.454498 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0d785-89c3-485d-b298-9514bc3c953b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:46.454498 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0d785-89c3-485d-b298-9514bc3c953b admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:46.454498 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e0d785-89c3-485d-b298-9514bc3c953b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:46.454858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25790669-7dd8-4b5a-8e03-3ebf7021107c None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:46.454938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e0d785-89c3-485d-b298-9514bc3c953b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:46.455026 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-25790669-7dd8-4b5a-8e03-3ebf7021107c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:46.463476 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 930/1866] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:46.464932 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0d785-89c3-485d-b298-9514bc3c953b admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:46.476332 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0d785-89c3-485d-b298-9514bc3c953b admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:46.482666 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50e23c-e688-4430-976e-32e2d4e9fe13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:46.485381 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50e23c-e688-4430-976e-32e2d4e9fe13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:46.485381 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50e23c-e688-4430-976e-32e2d4e9fe13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:46.485381 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50e23c-e688-4430-976e-32e2d4e9fe13 admin admin] PATH_INFO: `/v3/projects/1e2d7ad7477240128353e496566522d4/users/b465ef82dcd64ebda241be20c05ba344/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:46.501606 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50e23c-e688-4430-976e-32e2d4e9fe13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e2d7ad7477240128353e496566522d4, user_id=b465ef82dcd64ebda241be20c05ba344, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:46.504453 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50e23c-e688-4430-976e-32e2d4e9fe13 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:46.532772 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:46.540860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25790669-7dd8-4b5a-8e03-3ebf7021107c None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:46.547243 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c76f8-ac35-498d-83ff-562f3c8b16fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:46.548383 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c76f8-ac35-498d-83ff-562f3c8b16fd service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:46.548764 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c76f8-ac35-498d-83ff-562f3c8b16fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:46.549141 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c76f8-ac35-498d-83ff-562f3c8b16fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:46.560322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c76f8-ac35-498d-83ff-562f3c8b16fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:46.561934 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c76f8-ac35-498d-83ff-562f3c8b16fd service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:46.572811 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 937/1867] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:35:46.630283 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 931/1868] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:46 2022] PUT /identity/v3/projects/1e2d7ad7477240128353e496566522d4/users/b465ef82dcd64ebda241be20c05ba344/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:35:46.640919 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8af74-914a-4bd2-b92b-c15bcdfe344c None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:46.641722 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8af74-914a-4bd2-b92b-c15bcdfe344c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:46.641722 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8af74-914a-4bd2-b92b-c15bcdfe344c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:46.675081 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-e0b8af74-914a-4bd2-b92b-c15bcdfe344c None None] MFA Rules not processed for user `b465ef82dcd64ebda241be20c05ba344`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:46.676315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e0b8af74-914a-4bd2-b92b-c15bcdfe344c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:46.743390 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 938/1869] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:46.747771 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50e23c-e688-4430-976e-32e2d4e9fe13 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:46.824472 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50e23c-e688-4430-976e-32e2d4e9fe13 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:46.831276 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491542bf-6178-4e4e-8435-e3879ca9d262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:46.831276 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491542bf-6178-4e4e-8435-e3879ca9d262 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:46.831276 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491542bf-6178-4e4e-8435-e3879ca9d262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:46.831276 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491542bf-6178-4e4e-8435-e3879ca9d262 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:46.831908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491542bf-6178-4e4e-8435-e3879ca9d262 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:46.834337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491542bf-6178-4e4e-8435-e3879ca9d262 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:46.940815 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 932/1870] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:46 2022] POST /identity/v3/projects => generated 375 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:46.945080 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8af74-914a-4bd2-b92b-c15bcdfe344c None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:46.959256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8af74-914a-4bd2-b92b-c15bcdfe344c None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:47.059007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836feff-522e-41c1-b97a-f8ab51472f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:47.059817 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836feff-522e-41c1-b97a-f8ab51472f6a admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:47.060199 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836feff-522e-41c1-b97a-f8ab51472f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:47.060591 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836feff-522e-41c1-b97a-f8ab51472f6a admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:47.061715 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836feff-522e-41c1-b97a-f8ab51472f6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:47.064309 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836feff-522e-41c1-b97a-f8ab51472f6a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:47.158378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 939/1871] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:46 2022] POST /identity/v3/users => generated 294 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:47.162860 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491542bf-6178-4e4e-8435-e3879ca9d262 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:47.173267 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491542bf-6178-4e4e-8435-e3879ca9d262 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:47.178876 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4043deb-cf33-4996-b348-fb3a32f57885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:47.179546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4043deb-cf33-4996-b348-fb3a32f57885 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:47.179896 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4043deb-cf33-4996-b348-fb3a32f57885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:47.180091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4043deb-cf33-4996-b348-fb3a32f57885 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:47.180910 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4043deb-cf33-4996-b348-fb3a32f57885 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:47.182635 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4043deb-cf33-4996-b348-fb3a32f57885 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:47.198100 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 933/1872] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:35:47.199257 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836feff-522e-41c1-b97a-f8ab51472f6a admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:47.211239 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836feff-522e-41c1-b97a-f8ab51472f6a admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:47.216412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a83d0-7151-4ded-bc07-c6a3f61d425e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:47.217462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a83d0-7151-4ded-bc07-c6a3f61d425e admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:47.217747 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a83d0-7151-4ded-bc07-c6a3f61d425e admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:47.218018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a83d0-7151-4ded-bc07-c6a3f61d425e admin admin] PATH_INFO: `/v3/projects/43f139ff0fba435f84e6b5d203a57c54/users/2578e2a86ac04520847d3a888ca2e3e1/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:47.239052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639a83d0-7151-4ded-bc07-c6a3f61d425e admin admin] RBAC: Authorizing `identity:create_grant(project_id=43f139ff0fba435f84e6b5d203a57c54, user_id=2578e2a86ac04520847d3a888ca2e3e1, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:47.241742 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639a83d0-7151-4ded-bc07-c6a3f61d425e admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:47.256949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4043deb-cf33-4996-b348-fb3a32f57885 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:47.259484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d4043deb-cf33-4996-b348-fb3a32f57885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:47.278018 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4043deb-cf33-4996-b348-fb3a32f57885 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:47.283005 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e2053-ea41-46d0-a1db-90ba53ff35cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:47.283932 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e2053-ea41-46d0-a1db-90ba53ff35cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:47.284405 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e2053-ea41-46d0-a1db-90ba53ff35cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:47.284688 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e2053-ea41-46d0-a1db-90ba53ff35cf admin admin] PATH_INFO: `/v3/projects/4b070c4749c34407b659daf91ea6a30a` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:47.285916 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e2053-ea41-46d0-a1db-90ba53ff35cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b070c4749c34407b659daf91ea6a30a)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:47.288205 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e2053-ea41-46d0-a1db-90ba53ff35cf admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:47.356363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 940/1873] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:47 2022] PUT /identity/v3/projects/43f139ff0fba435f84e6b5d203a57c54/users/2578e2a86ac04520847d3a888ca2e3e1/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:35:47.360843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a83d0-7151-4ded-bc07-c6a3f61d425e admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:47.421716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a83d0-7151-4ded-bc07-c6a3f61d425e admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:47.426893 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be22e0-8fe1-43c4-bed6-1b5d7b94b55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:47.428076 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be22e0-8fe1-43c4-bed6-1b5d7b94b55d admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:47.428360 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be22e0-8fe1-43c4-bed6-1b5d7b94b55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:47.428634 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be22e0-8fe1-43c4-bed6-1b5d7b94b55d admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:47.429368 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be22e0-8fe1-43c4-bed6-1b5d7b94b55d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:47.431007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be22e0-8fe1-43c4-bed6-1b5d7b94b55d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:47.441292 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 941/1874] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:47.445850 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be22e0-8fe1-43c4-bed6-1b5d7b94b55d admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:47.450511 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-028e2053-ea41-46d0-a1db-90ba53ff35cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b070c4749c34407b659daf91ea6a30a', 'request_id': 'req-028e2053-ea41-46d0-a1db-90ba53ff35cf'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:47.458828 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be22e0-8fe1-43c4-bed6-1b5d7b94b55d admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:47.533758 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527b23c-6383-4bcc-a98a-b73015ea51b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:47.534622 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527b23c-6383-4bcc-a98a-b73015ea51b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:47.535295 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527b23c-6383-4bcc-a98a-b73015ea51b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:47.535295 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527b23c-6383-4bcc-a98a-b73015ea51b1 admin admin] PATH_INFO: `/v3/projects/43f139ff0fba435f84e6b5d203a57c54/users/2578e2a86ac04520847d3a888ca2e3e1/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:47.541835 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d527b23c-6383-4bcc-a98a-b73015ea51b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43f139ff0fba435f84e6b5d203a57c54, user_id=2578e2a86ac04520847d3a888ca2e3e1, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:47.544532 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d527b23c-6383-4bcc-a98a-b73015ea51b1 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:47.556118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 942/1875] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:47 2022] PUT /identity/v3/projects/43f139ff0fba435f84e6b5d203a57c54/users/2578e2a86ac04520847d3a888ca2e3e1/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:35:47.567180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f23a12-77f7-4d45-a36c-aacaffb4c5be None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:47.567475 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f23a12-77f7-4d45-a36c-aacaffb4c5be None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:47.568331 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f23a12-77f7-4d45-a36c-aacaffb4c5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:47.572487 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-028e2053-ea41-46d0-a1db-90ba53ff35cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b070c4749c34407b659daf91ea6a30a', 'request_id': 'req-028e2053-ea41-46d0-a1db-90ba53ff35cf'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:47.576516 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:47.580709 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-028e2053-ea41-46d0-a1db-90ba53ff35cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b070c4749c34407b659daf91ea6a30a', 'request_id': 'req-028e2053-ea41-46d0-a1db-90ba53ff35cf'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:47.591146 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-028e2053-ea41-46d0-a1db-90ba53ff35cf admin admin] The token cache is being invalidate because project 4b070c4749c34407b659daf91ea6a30a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:35:47.591436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-028e2053-ea41-46d0-a1db-90ba53ff35cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:47.592099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-028e2053-ea41-46d0-a1db-90ba53ff35cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:47.593667 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 934/1876] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:35:47 2022] DELETE /identity/v3/projects/4b070c4749c34407b659daf91ea6a30a => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:35:47.596566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-c5f23a12-77f7-4d45-a36c-aacaffb4c5be None None] MFA Rules not processed for user `2578e2a86ac04520847d3a888ca2e3e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:47.597703 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c5f23a12-77f7-4d45-a36c-aacaffb4c5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:47.597898 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e2053-ea41-46d0-a1db-90ba53ff35cf admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:47.599547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-028e2053-ea41-46d0-a1db-90ba53ff35cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:47.612602 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e2053-ea41-46d0-a1db-90ba53ff35cf admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:47.678028 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 943/1877] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:47 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:35:47.694139 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f23a12-77f7-4d45-a36c-aacaffb4c5be None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:47.694386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661c2b9-6ca0-46af-92ae-d5265287fd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:47.694857 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c5f23a12-77f7-4d45-a36c-aacaffb4c5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:47.694949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661c2b9-6ca0-46af-92ae-d5265287fd3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:47.694949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661c2b9-6ca0-46af-92ae-d5265287fd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:47.694949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661c2b9-6ca0-46af-92ae-d5265287fd3a admin admin] PATH_INFO: `/v3/users/d64b3bf418ae40cda178f693197a05d4` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:47.694949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0661c2b9-6ca0-46af-92ae-d5265287fd3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d64b3bf418ae40cda178f693197a05d4)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:47.696247 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0661c2b9-6ca0-46af-92ae-d5265287fd3a admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:47.749148 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f23a12-77f7-4d45-a36c-aacaffb4c5be None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:47.752809 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f6da1-5df4-45b5-8d5c-3da894e0f699 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:47.753482 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f6da1-5df4-45b5-8d5c-3da894e0f699 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:47.753767 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f6da1-5df4-45b5-8d5c-3da894e0f699 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:47.754049 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f6da1-5df4-45b5-8d5c-3da894e0f699 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:47.755937 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3e9f6da1-5df4-45b5-8d5c-3da894e0f699 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:47.815292 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f6da1-5df4-45b5-8d5c-3da894e0f699 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:47.816805 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f6da1-5df4-45b5-8d5c-3da894e0f699 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:47.826824 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 944/1878] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:35:47.903023 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0661c2b9-6ca0-46af-92ae-d5265287fd3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd64b3bf418ae40cda178f693197a05d4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:47.975862 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0661c2b9-6ca0-46af-92ae-d5265287fd3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd64b3bf418ae40cda178f693197a05d4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:47.976491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0661c2b9-6ca0-46af-92ae-d5265287fd3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd64b3bf418ae40cda178f693197a05d4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:47.987839 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0661c2b9-6ca0-46af-92ae-d5265287fd3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd64b3bf418ae40cda178f693197a05d4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:47.987839 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0661c2b9-6ca0-46af-92ae-d5265287fd3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd64b3bf418ae40cda178f693197a05d4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:48.010653 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 935/1879] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:35:47 2022] DELETE /identity/v3/users/d64b3bf418ae40cda178f693197a05d4 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:48.528405 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f6da1-5df4-45b5-8d5c-3da894e0f699 service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:48.539183 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f6da1-5df4-45b5-8d5c-3da894e0f699 service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:48.550993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:48.666754 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941d009a-a461-46c7-8db2-af0b5e8a61a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:48.668254 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d009a-a461-46c7-8db2-af0b5e8a61a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:48.668555 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d009a-a461-46c7-8db2-af0b5e8a61a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:48.668842 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d009a-a461-46c7-8db2-af0b5e8a61a5 admin admin] PATH_INFO: `/v3/projects/63088692fa9540dd843c2c50ad945d8c` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:48.672308 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941d009a-a461-46c7-8db2-af0b5e8a61a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63088692fa9540dd843c2c50ad945d8c)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:48.674297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941d009a-a461-46c7-8db2-af0b5e8a61a5 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:48.826587 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-941d009a-a461-46c7-8db2-af0b5e8a61a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63088692fa9540dd843c2c50ad945d8c', 'request_id': 'req-941d009a-a461-46c7-8db2-af0b5e8a61a5'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:48.935139 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-941d009a-a461-46c7-8db2-af0b5e8a61a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63088692fa9540dd843c2c50ad945d8c', 'request_id': 'req-941d009a-a461-46c7-8db2-af0b5e8a61a5'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:48.941849 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-941d009a-a461-46c7-8db2-af0b5e8a61a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63088692fa9540dd843c2c50ad945d8c', 'request_id': 'req-941d009a-a461-46c7-8db2-af0b5e8a61a5'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:48.954810 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-941d009a-a461-46c7-8db2-af0b5e8a61a5 admin admin] The token cache is being invalidate because project 63088692fa9540dd843c2c50ad945d8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:35:48.955122 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-941d009a-a461-46c7-8db2-af0b5e8a61a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:48.955752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-941d009a-a461-46c7-8db2-af0b5e8a61a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:48.957314 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 945/1880] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:35:48 2022] DELETE /identity/v3/projects/63088692fa9540dd843c2c50ad945d8c => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:48.963599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c12f72-10cf-40cc-b13a-22519d60c0ae None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:48.964116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c12f72-10cf-40cc-b13a-22519d60c0ae None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:48.964211 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c12f72-10cf-40cc-b13a-22519d60c0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:48.984653 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-23c12f72-10cf-40cc-b13a-22519d60c0ae None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:48.986120 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-23c12f72-10cf-40cc-b13a-22519d60c0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:49.044861 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 936/1881] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:49.053066 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21812f61-c7d3-416a-879b-068650367235 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:49.053350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21812f61-c7d3-416a-879b-068650367235 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:49.053623 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21812f61-c7d3-416a-879b-068650367235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:49.072706 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-21812f61-c7d3-416a-879b-068650367235 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:49.073849 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-21812f61-c7d3-416a-879b-068650367235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:49.083108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 946/1882] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:35:49.089925 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c12f72-10cf-40cc-b13a-22519d60c0ae None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:49.097072 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c12f72-10cf-40cc-b13a-22519d60c0ae None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:49.102022 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930465c1-079f-4e3a-918d-b6b96962b65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:49.102759 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930465c1-079f-4e3a-918d-b6b96962b65f admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:49.103139 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930465c1-079f-4e3a-918d-b6b96962b65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:49.103489 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930465c1-079f-4e3a-918d-b6b96962b65f admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:49.104302 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930465c1-079f-4e3a-918d-b6b96962b65f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:35:49.104611 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930465c1-079f-4e3a-918d-b6b96962b65f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:49.106208 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930465c1-079f-4e3a-918d-b6b96962b65f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:49.128384 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 937/1883] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:35:49.132410 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21812f61-c7d3-416a-879b-068650367235 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:49.143752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21812f61-c7d3-416a-879b-068650367235 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:49.149551 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec67d45-1c16-4a5c-a5e0-da327b7f59ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:49.150218 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec67d45-1c16-4a5c-a5e0-da327b7f59ff admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:49.150496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec67d45-1c16-4a5c-a5e0-da327b7f59ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:49.150769 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec67d45-1c16-4a5c-a5e0-da327b7f59ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:49.151503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec67d45-1c16-4a5c-a5e0-da327b7f59ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:49.153665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec67d45-1c16-4a5c-a5e0-da327b7f59ff admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:49.307560 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 947/1884] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:49 2022] POST /identity/v3/projects => generated 381 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:49.318053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930465c1-079f-4e3a-918d-b6b96962b65f admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:49.334409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930465c1-079f-4e3a-918d-b6b96962b65f admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:49.415568 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a0e7d-121d-44d6-90a7-cddfebd9ef6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:49.416248 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a0e7d-121d-44d6-90a7-cddfebd9ef6f admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:49.416530 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a0e7d-121d-44d6-90a7-cddfebd9ef6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:49.416865 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83a0e7d-121d-44d6-90a7-cddfebd9ef6f admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:49.417558 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83a0e7d-121d-44d6-90a7-cddfebd9ef6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:49.422074 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83a0e7d-121d-44d6-90a7-cddfebd9ef6f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:49.553654 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:49.560839 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 938/1885] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:49 2022] POST /identity/v3/users => generated 291 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:49.565230 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec67d45-1c16-4a5c-a5e0-da327b7f59ff admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:49.576572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec67d45-1c16-4a5c-a5e0-da327b7f59ff admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:49.581237 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58d622-51b5-47d2-adb2-9f5109987ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:49.581938 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58d622-51b5-47d2-adb2-9f5109987ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:49.582219 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58d622-51b5-47d2-adb2-9f5109987ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:49.582494 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58d622-51b5-47d2-adb2-9f5109987ab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:49.583809 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58d622-51b5-47d2-adb2-9f5109987ab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:49.586176 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58d622-51b5-47d2-adb2-9f5109987ab2 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:49.596902 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 948/1886] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:49.603229 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a0e7d-121d-44d6-90a7-cddfebd9ef6f admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:49.622374 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83a0e7d-121d-44d6-90a7-cddfebd9ef6f admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:49.625282 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:49.631461 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ee54a-5aae-44bd-9864-550aba32c2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:49.633843 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ee54a-5aae-44bd-9864-550aba32c2a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:49.634297 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ee54a-5aae-44bd-9864-550aba32c2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:49.634424 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ee54a-5aae-44bd-9864-550aba32c2a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:49.635155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ee54a-5aae-44bd-9864-550aba32c2a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:49.636779 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ee54a-5aae-44bd-9864-550aba32c2a2 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:49.651540 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 939/1887] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:49.652317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58d622-51b5-47d2-adb2-9f5109987ab2 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:49.675789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58d622-51b5-47d2-adb2-9f5109987ab2 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:49.680823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4dab6-a7fd-4203-bedf-1be017e38adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:49.690128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4dab6-a7fd-4203-bedf-1be017e38adf admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:49.690128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4dab6-a7fd-4203-bedf-1be017e38adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:49.690128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4dab6-a7fd-4203-bedf-1be017e38adf admin admin] PATH_INFO: `/v3/projects/625821c00c8d4c9f843e86386e4c61e5/users/4eb23449930743058cca3969a20a3b51/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:49.706009 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4dab6-a7fd-4203-bedf-1be017e38adf admin admin] RBAC: Authorizing `identity:create_grant(project_id=625821c00c8d4c9f843e86386e4c61e5, user_id=4eb23449930743058cca3969a20a3b51, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:49.712513 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4dab6-a7fd-4203-bedf-1be017e38adf admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:49.830011 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 949/1888] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:49 2022] PUT /identity/v3/projects/625821c00c8d4c9f843e86386e4c61e5/users/4eb23449930743058cca3969a20a3b51/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:35:49.841740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858bf4d5-fd53-407a-8f64-70ad836de5d5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:49.841986 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858bf4d5-fd53-407a-8f64-70ad836de5d5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:49.842263 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858bf4d5-fd53-407a-8f64-70ad836de5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:49.871075 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-858bf4d5-fd53-407a-8f64-70ad836de5d5 None None] MFA Rules not processed for user `4eb23449930743058cca3969a20a3b51`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:49.872667 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-858bf4d5-fd53-407a-8f64-70ad836de5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:49.946504 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 940/1889] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:49.951121 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4dab6-a7fd-4203-bedf-1be017e38adf admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:50.022492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4dab6-a7fd-4203-bedf-1be017e38adf admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:50.028060 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4ac51-8cc4-4f16-a11e-3a382803adc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:50.028974 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4ac51-8cc4-4f16-a11e-3a382803adc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:50.029333 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4ac51-8cc4-4f16-a11e-3a382803adc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:50.029678 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4ac51-8cc4-4f16-a11e-3a382803adc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:50.030456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc4ac51-8cc4-4f16-a11e-3a382803adc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:50.032545 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc4ac51-8cc4-4f16-a11e-3a382803adc0 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:50.110174 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 950/1890] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:49 2022] POST /identity/v3/projects => generated 381 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:50.112559 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858bf4d5-fd53-407a-8f64-70ad836de5d5 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:50.125754 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858bf4d5-fd53-407a-8f64-70ad836de5d5 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:50.185344 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003e5f2-9e58-4009-b631-743a6e24b0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:50.186025 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003e5f2-9e58-4009-b631-743a6e24b0e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:50.186308 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003e5f2-9e58-4009-b631-743a6e24b0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:50.186586 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003e5f2-9e58-4009-b631-743a6e24b0e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:50.187321 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6003e5f2-9e58-4009-b631-743a6e24b0e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:50.189451 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6003e5f2-9e58-4009-b631-743a6e24b0e7 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:50.323939 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 941/1891] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:50 2022] POST /identity/v3/users => generated 297 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:50.333902 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4ac51-8cc4-4f16-a11e-3a382803adc0 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:50.347971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4ac51-8cc4-4f16-a11e-3a382803adc0 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:50.360779 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e2591-539b-4b5a-bc4a-79ec39e2c805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:50.360779 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e2591-539b-4b5a-bc4a-79ec39e2c805 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:50.360779 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e2591-539b-4b5a-bc4a-79ec39e2c805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:50.360779 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e2591-539b-4b5a-bc4a-79ec39e2c805 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:50.361150 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e2591-539b-4b5a-bc4a-79ec39e2c805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:50.361150 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e2591-539b-4b5a-bc4a-79ec39e2c805 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:50.399027 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 951/1892] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:50.400336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003e5f2-9e58-4009-b631-743a6e24b0e7 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:50.414883 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003e5f2-9e58-4009-b631-743a6e24b0e7 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:50.423276 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102581b6-a3a5-4a1d-9d9f-a0f08e5e4612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:50.424247 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102581b6-a3a5-4a1d-9d9f-a0f08e5e4612 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:50.424532 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102581b6-a3a5-4a1d-9d9f-a0f08e5e4612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:50.424807 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102581b6-a3a5-4a1d-9d9f-a0f08e5e4612 admin admin] PATH_INFO: `/v3/projects/1bf7812778274edea98c8e09cd5522dd/users/4922c7269260456eb5d24d1388f1f995/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:50.442405 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102581b6-a3a5-4a1d-9d9f-a0f08e5e4612 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bf7812778274edea98c8e09cd5522dd, user_id=4922c7269260456eb5d24d1388f1f995, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:50.445831 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102581b6-a3a5-4a1d-9d9f-a0f08e5e4612 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:50.600329 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 942/1893] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:50 2022] PUT /identity/v3/projects/1bf7812778274edea98c8e09cd5522dd/users/4922c7269260456eb5d24d1388f1f995/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:50.609634 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e2591-539b-4b5a-bc4a-79ec39e2c805 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:50.619828 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e2591-539b-4b5a-bc4a-79ec39e2c805 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:50.678556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1ce0e-607b-4c50-b15d-28ffa6337f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:50.679368 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1ce0e-607b-4c50-b15d-28ffa6337f0c admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:50.679651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1ce0e-607b-4c50-b15d-28ffa6337f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:50.679932 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1ce0e-607b-4c50-b15d-28ffa6337f0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:50.680854 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1ce0e-607b-4c50-b15d-28ffa6337f0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:50.682669 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1ce0e-607b-4c50-b15d-28ffa6337f0c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:50.693371 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 952/1894] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:50 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:50.697410 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102581b6-a3a5-4a1d-9d9f-a0f08e5e4612 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:50.708181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102581b6-a3a5-4a1d-9d9f-a0f08e5e4612 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:50.712938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48a0bd-b7b4-4b77-9074-b94da8b748de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:50.713897 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48a0bd-b7b4-4b77-9074-b94da8b748de admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:50.714188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48a0bd-b7b4-4b77-9074-b94da8b748de admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:50.714468 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48a0bd-b7b4-4b77-9074-b94da8b748de admin admin] PATH_INFO: `/v3/projects/1bf7812778274edea98c8e09cd5522dd/users/4922c7269260456eb5d24d1388f1f995/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:50.716525 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca48a0bd-b7b4-4b77-9074-b94da8b748de admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bf7812778274edea98c8e09cd5522dd, user_id=4922c7269260456eb5d24d1388f1f995, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:50.719214 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca48a0bd-b7b4-4b77-9074-b94da8b748de admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:50.730733 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 943/1895] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:50 2022] PUT /identity/v3/projects/1bf7812778274edea98c8e09cd5522dd/users/4922c7269260456eb5d24d1388f1f995/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:50.742992 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6e6d4-8d3b-493d-9502-e6939d887d21 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:50.743288 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6e6d4-8d3b-493d-9502-e6939d887d21 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:50.743566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b6e6d4-8d3b-493d-9502-e6939d887d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:50.774496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-c5b6e6d4-8d3b-493d-9502-e6939d887d21 None None] MFA Rules not processed for user `4922c7269260456eb5d24d1388f1f995`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:50.779780 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c5b6e6d4-8d3b-493d-9502-e6939d887d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:50.818441 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48a0bd-b7b4-4b77-9074-b94da8b748de admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:50.820300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ca48a0bd-b7b4-4b77-9074-b94da8b748de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:50.825393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:50.904463 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 953/1896] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:50.904794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48a0bd-b7b4-4b77-9074-b94da8b748de admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:50.920204 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019115f5-b4d7-4853-83a9-37f0f85b2e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:50.920204 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019115f5-b4d7-4853-83a9-37f0f85b2e33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:50.920204 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019115f5-b4d7-4853-83a9-37f0f85b2e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:50.920204 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019115f5-b4d7-4853-83a9-37f0f85b2e33 admin admin] PATH_INFO: `/v3/users/0433dab9503b44ae9aceb6330aa86b02` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:50.920572 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019115f5-b4d7-4853-83a9-37f0f85b2e33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0433dab9503b44ae9aceb6330aa86b02)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:50.920572 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019115f5-b4d7-4853-83a9-37f0f85b2e33 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:50.920742 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6e6d4-8d3b-493d-9502-e6939d887d21 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:50.920742 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c5b6e6d4-8d3b-493d-9502-e6939d887d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:50.976275 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b6e6d4-8d3b-493d-9502-e6939d887d21 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:50.979884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f457e-e471-4b6d-9fe0-fc2ec07dc177 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:50.980528 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f457e-e471-4b6d-9fe0-fc2ec07dc177 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:50.980806 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f457e-e471-4b6d-9fe0-fc2ec07dc177 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:50.981078 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f457e-e471-4b6d-9fe0-fc2ec07dc177 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:51.035402 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f457e-e471-4b6d-9fe0-fc2ec07dc177 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:51.037443 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f457e-e471-4b6d-9fe0-fc2ec07dc177 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:51.046390 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 954/1897] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:51.103174 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-019115f5-b4d7-4853-83a9-37f0f85b2e33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0433dab9503b44ae9aceb6330aa86b02'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:51.157805 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-019115f5-b4d7-4853-83a9-37f0f85b2e33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0433dab9503b44ae9aceb6330aa86b02'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:51.158472 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-019115f5-b4d7-4853-83a9-37f0f85b2e33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0433dab9503b44ae9aceb6330aa86b02'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:51.166782 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-019115f5-b4d7-4853-83a9-37f0f85b2e33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0433dab9503b44ae9aceb6330aa86b02'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:51.167816 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-019115f5-b4d7-4853-83a9-37f0f85b2e33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0433dab9503b44ae9aceb6330aa86b02'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:51.185356 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 944/1898] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:35:50 2022] DELETE /identity/v3/users/0433dab9503b44ae9aceb6330aa86b02 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:51.189647 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0976e-2b69-4b63-a743-98a51013a4b4 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:51.190769 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0976e-2b69-4b63-a743-98a51013a4b4 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:51.190769 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0976e-2b69-4b63-a743-98a51013a4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:51.208960 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-a3c0976e-2b69-4b63-a743-98a51013a4b4 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:51.210205 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a3c0976e-2b69-4b63-a743-98a51013a4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:51.275196 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 955/1899] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:51.284953 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019115f5-b4d7-4853-83a9-37f0f85b2e33 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:51.287040 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-019115f5-b4d7-4853-83a9-37f0f85b2e33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:51.345844 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019115f5-b4d7-4853-83a9-37f0f85b2e33 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:51.352223 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ca421-7a4c-4a3b-b2f1-23c467c88178 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:51.352888 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ca421-7a4c-4a3b-b2f1-23c467c88178 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:51.353178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ca421-7a4c-4a3b-b2f1-23c467c88178 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:51.353450 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ca421-7a4c-4a3b-b2f1-23c467c88178 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:51.363429 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ca421-7a4c-4a3b-b2f1-23c467c88178 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:51.368146 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ca421-7a4c-4a3b-b2f1-23c467c88178 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:51.374798 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 945/1900] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:51.953821 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0976e-2b69-4b63-a743-98a51013a4b4 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:51.955627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a3c0976e-2b69-4b63-a743-98a51013a4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:51.990174 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0976e-2b69-4b63-a743-98a51013a4b4 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:52.004211 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564c410-7b45-47ae-a35b-94159cf3ac5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:52.008144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564c410-7b45-47ae-a35b-94159cf3ac5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:52.008144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564c410-7b45-47ae-a35b-94159cf3ac5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:52.008144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564c410-7b45-47ae-a35b-94159cf3ac5d admin admin] PATH_INFO: `/v3/projects/17fa13779428430d93fd420554c9f213` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:52.008771 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1564c410-7b45-47ae-a35b-94159cf3ac5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=17fa13779428430d93fd420554c9f213)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:52.010757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1564c410-7b45-47ae-a35b-94159cf3ac5d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:52.136136 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1564c410-7b45-47ae-a35b-94159cf3ac5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17fa13779428430d93fd420554c9f213', 'request_id': 'req-1564c410-7b45-47ae-a35b-94159cf3ac5d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:52.398174 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:52.426325 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1564c410-7b45-47ae-a35b-94159cf3ac5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17fa13779428430d93fd420554c9f213', 'request_id': 'req-1564c410-7b45-47ae-a35b-94159cf3ac5d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:52.433016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1564c410-7b45-47ae-a35b-94159cf3ac5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17fa13779428430d93fd420554c9f213', 'request_id': 'req-1564c410-7b45-47ae-a35b-94159cf3ac5d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:52.444594 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1564c410-7b45-47ae-a35b-94159cf3ac5d admin admin] The token cache is being invalidate because project 17fa13779428430d93fd420554c9f213 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:35:52.444892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1564c410-7b45-47ae-a35b-94159cf3ac5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:52.445557 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1564c410-7b45-47ae-a35b-94159cf3ac5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:52.447150 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 956/1901] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:35:51 2022] DELETE /identity/v3/projects/17fa13779428430d93fd420554c9f213 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:52.451456 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ca421-7a4c-4a3b-b2f1-23c467c88178 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:52.461045 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ca421-7a4c-4a3b-b2f1-23c467c88178 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:52.514879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3ffeb-8c35-4ad9-bbf0-ba681fbe48c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:52.515734 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3ffeb-8c35-4ad9-bbf0-ba681fbe48c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:52.516012 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3ffeb-8c35-4ad9-bbf0-ba681fbe48c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:52.516313 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3ffeb-8c35-4ad9-bbf0-ba681fbe48c4 admin admin] PATH_INFO: `/v3/users/1ef9345e8e5b41f59ce8492bb6dfe216` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:52.517481 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf3ffeb-8c35-4ad9-bbf0-ba681fbe48c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ef9345e8e5b41f59ce8492bb6dfe216)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:52.519439 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf3ffeb-8c35-4ad9-bbf0-ba681fbe48c4 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:52.714557 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ecf3ffeb-8c35-4ad9-bbf0-ba681fbe48c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ef9345e8e5b41f59ce8492bb6dfe216'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:52.803736 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ecf3ffeb-8c35-4ad9-bbf0-ba681fbe48c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ef9345e8e5b41f59ce8492bb6dfe216'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:52.804361 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ecf3ffeb-8c35-4ad9-bbf0-ba681fbe48c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ef9345e8e5b41f59ce8492bb6dfe216'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:52.811839 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ecf3ffeb-8c35-4ad9-bbf0-ba681fbe48c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ef9345e8e5b41f59ce8492bb6dfe216'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:52.812513 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ecf3ffeb-8c35-4ad9-bbf0-ba681fbe48c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ef9345e8e5b41f59ce8492bb6dfe216'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:52.830335 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 946/1902] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:35:52 2022] DELETE /identity/v3/users/1ef9345e8e5b41f59ce8492bb6dfe216 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:53.118143 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:53.308553 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564c410-7b45-47ae-a35b-94159cf3ac5d admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:53.310068 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1564c410-7b45-47ae-a35b-94159cf3ac5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:53.323566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564c410-7b45-47ae-a35b-94159cf3ac5d admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:53.382498 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a4b7c-6079-4475-bcff-4e477d2be7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:53.383458 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a4b7c-6079-4475-bcff-4e477d2be7c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:53.383734 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a4b7c-6079-4475-bcff-4e477d2be7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:53.384007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a4b7c-6079-4475-bcff-4e477d2be7c1 admin admin] PATH_INFO: `/v3/projects/38a9eaf2450e4de581fcc2c8be087fd8` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:53.385180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a4b7c-6079-4475-bcff-4e477d2be7c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38a9eaf2450e4de581fcc2c8be087fd8)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:53.387110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a4b7c-6079-4475-bcff-4e477d2be7c1 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:53.515308 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-9c6a4b7c-6079-4475-bcff-4e477d2be7c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38a9eaf2450e4de581fcc2c8be087fd8', 'request_id': 'req-9c6a4b7c-6079-4475-bcff-4e477d2be7c1'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:53.600920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-9c6a4b7c-6079-4475-bcff-4e477d2be7c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38a9eaf2450e4de581fcc2c8be087fd8', 'request_id': 'req-9c6a4b7c-6079-4475-bcff-4e477d2be7c1'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:53.606619 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-9c6a4b7c-6079-4475-bcff-4e477d2be7c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38a9eaf2450e4de581fcc2c8be087fd8', 'request_id': 'req-9c6a4b7c-6079-4475-bcff-4e477d2be7c1'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:53.618010 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-9c6a4b7c-6079-4475-bcff-4e477d2be7c1 admin admin] The token cache is being invalidate because project 38a9eaf2450e4de581fcc2c8be087fd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:35:53.618297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-9c6a4b7c-6079-4475-bcff-4e477d2be7c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:53.618955 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-9c6a4b7c-6079-4475-bcff-4e477d2be7c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:35:53.621232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 957/1903] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:35:53 2022] DELETE /identity/v3/projects/38a9eaf2450e4de581fcc2c8be087fd8 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:53.627602 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e423476-ec98-4a02-a529-d084aa42a6ae None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:53.628075 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e423476-ec98-4a02-a529-d084aa42a6ae None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:53.628152 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e423476-ec98-4a02-a529-d084aa42a6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:53.647245 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-3e423476-ec98-4a02-a529-d084aa42a6ae None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:53.648448 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3e423476-ec98-4a02-a529-d084aa42a6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:53.706184 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 947/1904] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:53.714462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7e0da-e7f0-4f91-92a9-1f62aa97481d None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:53.714770 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7e0da-e7f0-4f91-92a9-1f62aa97481d None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:53.715078 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7e0da-e7f0-4f91-92a9-1f62aa97481d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:53.732899 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-61b7e0da-e7f0-4f91-92a9-1f62aa97481d None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:53.734105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-61b7e0da-e7f0-4f91-92a9-1f62aa97481d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:53.741894 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 958/1905] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:53.745983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e423476-ec98-4a02-a529-d084aa42a6ae None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:53.756337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e423476-ec98-4a02-a529-d084aa42a6ae None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:53.760957 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621acd8-b7c7-4ef3-b6e6-93b363122476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:53.761602 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621acd8-b7c7-4ef3-b6e6-93b363122476 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:53.761886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621acd8-b7c7-4ef3-b6e6-93b363122476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:53.762153 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621acd8-b7c7-4ef3-b6e6-93b363122476 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:53.762952 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621acd8-b7c7-4ef3-b6e6-93b363122476 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:35:53.763258 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621acd8-b7c7-4ef3-b6e6-93b363122476 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:53.764844 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621acd8-b7c7-4ef3-b6e6-93b363122476 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:53.776929 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 948/1906] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:35:53.781515 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7e0da-e7f0-4f91-92a9-1f62aa97481d None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:53.799447 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7e0da-e7f0-4f91-92a9-1f62aa97481d None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:53.799447 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb3140-e10b-4307-bdd4-265d3db1c635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:53.799447 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb3140-e10b-4307-bdd4-265d3db1c635 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:53.799447 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb3140-e10b-4307-bdd4-265d3db1c635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:53.799782 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eb3140-e10b-4307-bdd4-265d3db1c635 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:53.799782 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eb3140-e10b-4307-bdd4-265d3db1c635 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:53.800868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eb3140-e10b-4307-bdd4-265d3db1c635 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:53.925104 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 959/1907] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:35:53 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:53.929678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621acd8-b7c7-4ef3-b6e6-93b363122476 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:53.939726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621acd8-b7c7-4ef3-b6e6-93b363122476 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:53.995790 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf419b-2e19-4488-9be5-a84ed2172de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:53.996551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf419b-2e19-4488-9be5-a84ed2172de5 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:53.997088 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf419b-2e19-4488-9be5-a84ed2172de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:53.997610 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf419b-2e19-4488-9be5-a84ed2172de5 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:53.999421 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:53.999940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf419b-2e19-4488-9be5-a84ed2172de5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:54.001912 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf419b-2e19-4488-9be5-a84ed2172de5 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:54.113027 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 949/1908] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:35:53 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:35:54.117012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb3140-e10b-4307-bdd4-265d3db1c635 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:54.127305 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eb3140-e10b-4307-bdd4-265d3db1c635 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:54.133100 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829543a-3923-4ee5-98ae-9a98df2fa127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:54.134028 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829543a-3923-4ee5-98ae-9a98df2fa127 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:54.135291 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829543a-3923-4ee5-98ae-9a98df2fa127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:54.135659 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829543a-3923-4ee5-98ae-9a98df2fa127 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:54.136861 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a829543a-3923-4ee5-98ae-9a98df2fa127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:54.138539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a829543a-3923-4ee5-98ae-9a98df2fa127 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:54.153477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 960/1909] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:54.155130 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf419b-2e19-4488-9be5-a84ed2172de5 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:54.165522 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf419b-2e19-4488-9be5-a84ed2172de5 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:54.170529 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1ee7d-aba8-4900-be92-b426d28f9f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:54.174589 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1ee7d-aba8-4900-be92-b426d28f9f27 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:54.174589 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1ee7d-aba8-4900-be92-b426d28f9f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:54.174589 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1ee7d-aba8-4900-be92-b426d28f9f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:54.174589 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba1ee7d-aba8-4900-be92-b426d28f9f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:54.174589 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba1ee7d-aba8-4900-be92-b426d28f9f27 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:54.184750 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 950/1910] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:35:54.188918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829543a-3923-4ee5-98ae-9a98df2fa127 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:54.200007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829543a-3923-4ee5-98ae-9a98df2fa127 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:54.205037 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a0825-6379-440e-901e-af71417fe8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:54.205907 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a0825-6379-440e-901e-af71417fe8a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:54.206184 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a0825-6379-440e-901e-af71417fe8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:54.206454 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a0825-6379-440e-901e-af71417fe8a6 admin admin] PATH_INFO: `/v3/projects/dc902dcd5bcd47c1bb3f4e87496fcec0/users/12e541a7d9454807a366890942bee2f3/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:54.221856 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a0825-6379-440e-901e-af71417fe8a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc902dcd5bcd47c1bb3f4e87496fcec0, user_id=12e541a7d9454807a366890942bee2f3, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:54.224947 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a0825-6379-440e-901e-af71417fe8a6 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:54.387739 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 961/1911] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:35:54 2022] PUT /identity/v3/projects/dc902dcd5bcd47c1bb3f4e87496fcec0/users/12e541a7d9454807a366890942bee2f3/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:35:54.398376 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb11ff7-a416-43a3-94e1-d82f50f6f10a None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:54.398746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb11ff7-a416-43a3-94e1-d82f50f6f10a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:54.399126 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb11ff7-a416-43a3-94e1-d82f50f6f10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:54.415466 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:54.427616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-6fb11ff7-a416-43a3-94e1-d82f50f6f10a None None] MFA Rules not processed for user `12e541a7d9454807a366890942bee2f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:35:54.429178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6fb11ff7-a416-43a3-94e1-d82f50f6f10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:54.496161 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 951/1912] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:35:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:35:54.506915 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a0825-6379-440e-901e-af71417fe8a6 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:54.508648 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6e3a0825-6379-440e-901e-af71417fe8a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:54.569534 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a0825-6379-440e-901e-af71417fe8a6 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:54.573496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab766b7-5511-4da8-af55-9b00235225a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:54.574147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab766b7-5511-4da8-af55-9b00235225a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:54.574428 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab766b7-5511-4da8-af55-9b00235225a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:54.574702 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab766b7-5511-4da8-af55-9b00235225a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:54.583351 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab766b7-5511-4da8-af55-9b00235225a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:54.584785 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab766b7-5511-4da8-af55-9b00235225a7 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:54.593398 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 962/1913] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:35:55.126108 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb11ff7-a416-43a3-94e1-d82f50f6f10a None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:35:55.127591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6fb11ff7-a416-43a3-94e1-d82f50f6f10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:55.139187 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb11ff7-a416-43a3-94e1-d82f50f6f10a None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:35:55.142679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370e583-d93b-4298-9fe9-12dfa225424b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:35:55.143344 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370e583-d93b-4298-9fe9-12dfa225424b service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:35:55.143624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370e583-d93b-4298-9fe9-12dfa225424b service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:35:55.143894 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370e583-d93b-4298-9fe9-12dfa225424b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:35:55.145634 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f370e583-d93b-4298-9fe9-12dfa225424b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:35:55.236093 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f370e583-d93b-4298-9fe9-12dfa225424b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:35:55.237698 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f370e583-d93b-4298-9fe9-12dfa225424b service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:35:55.250468 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 952/1914] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:35:55 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:55.352760 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:55.553713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:56.435230 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:56.672374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:56.955619 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:57.139242 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:57.664283 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:57.692490 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:58.018370 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:58.053151 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:58.403949 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:58.532573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:58.557987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:58.706728 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:58.738133 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:58.820622 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:58.849761 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:58.995231 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:09.024169 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:09.109548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=85,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:13.635085 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab766b7-5511-4da8-af55-9b00235225a7 service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:13.643648 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab766b7-5511-4da8-af55-9b00235225a7 service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:13.648489 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6f1647-6da1-494d-ba97-4252b03141d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:13.649125 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f1647-6da1-494d-ba97-4252b03141d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:13.649405 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f1647-6da1-494d-ba97-4252b03141d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:13.649679 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6f1647-6da1-494d-ba97-4252b03141d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:13.651478 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4d6f1647-6da1-494d-ba97-4252b03141d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:13.714541 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f1647-6da1-494d-ba97-4252b03141d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:13.716016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6f1647-6da1-494d-ba97-4252b03141d1 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:13.726166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 963/1915] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:36:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:21.214786 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370e583-d93b-4298-9fe9-12dfa225424b service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:21.286935 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370e583-d93b-4298-9fe9-12dfa225424b service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:21.296271 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633dcea0-995f-406b-a71e-9624cc677542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:21.297136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633dcea0-995f-406b-a71e-9624cc677542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:21.297422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633dcea0-995f-406b-a71e-9624cc677542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:21.297833 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633dcea0-995f-406b-a71e-9624cc677542 admin admin] PATH_INFO: `/v3/users/12e541a7d9454807a366890942bee2f3` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:21.299218 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633dcea0-995f-406b-a71e-9624cc677542 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12e541a7d9454807a366890942bee2f3)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:21.303107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633dcea0-995f-406b-a71e-9624cc677542 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:21.555690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-633dcea0-995f-406b-a71e-9624cc677542 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12e541a7d9454807a366890942bee2f3'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:21.635506 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-633dcea0-995f-406b-a71e-9624cc677542 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12e541a7d9454807a366890942bee2f3'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:21.636221 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-633dcea0-995f-406b-a71e-9624cc677542 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12e541a7d9454807a366890942bee2f3'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:21.644452 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-633dcea0-995f-406b-a71e-9624cc677542 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12e541a7d9454807a366890942bee2f3'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:21.645139 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-633dcea0-995f-406b-a71e-9624cc677542 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12e541a7d9454807a366890942bee2f3'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:21.670118 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 953/1916] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:36:21 2022] DELETE /identity/v3/users/12e541a7d9454807a366890942bee2f3 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:21.671580 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ab486-fc13-4d3c-9974-70adf57fdc6b None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:21.672076 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ab486-fc13-4d3c-9974-70adf57fdc6b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:21.672360 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ab486-fc13-4d3c-9974-70adf57fdc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:21.691486 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-176ab486-fc13-4d3c-9974-70adf57fdc6b None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:21.693021 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-176ab486-fc13-4d3c-9974-70adf57fdc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:21.761253 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 964/1917] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:21.785645 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633dcea0-995f-406b-a71e-9624cc677542 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:21.787309 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-633dcea0-995f-406b-a71e-9624cc677542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:21.851721 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633dcea0-995f-406b-a71e-9624cc677542 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:21.855442 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa6cf2-dadd-4790-9291-f0198b059cb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:21.856091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa6cf2-dadd-4790-9291-f0198b059cb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:21.856368 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa6cf2-dadd-4790-9291-f0198b059cb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:21.856638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa6cf2-dadd-4790-9291-f0198b059cb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:21.867964 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa6cf2-dadd-4790-9291-f0198b059cb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:21.869448 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa6cf2-dadd-4790-9291-f0198b059cb0 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:21.880322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 954/1918] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:36:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:22.464441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ab486-fc13-4d3c-9974-70adf57fdc6b None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:22.465940 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-176ab486-fc13-4d3c-9974-70adf57fdc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:22.478550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ab486-fc13-4d3c-9974-70adf57fdc6b None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:22.483962 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b62e98-4e87-43f6-aa8b-0863dd781673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:22.484863 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b62e98-4e87-43f6-aa8b-0863dd781673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:22.485129 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b62e98-4e87-43f6-aa8b-0863dd781673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:22.485570 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b62e98-4e87-43f6-aa8b-0863dd781673 admin admin] PATH_INFO: `/v3/projects/dc902dcd5bcd47c1bb3f4e87496fcec0` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:22.486893 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b62e98-4e87-43f6-aa8b-0863dd781673 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc902dcd5bcd47c1bb3f4e87496fcec0)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:22.488878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b62e98-4e87-43f6-aa8b-0863dd781673 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:22.642054 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-69b62e98-4e87-43f6-aa8b-0863dd781673 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc902dcd5bcd47c1bb3f4e87496fcec0', 'request_id': 'req-69b62e98-4e87-43f6-aa8b-0863dd781673'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:22.735343 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-69b62e98-4e87-43f6-aa8b-0863dd781673 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc902dcd5bcd47c1bb3f4e87496fcec0', 'request_id': 'req-69b62e98-4e87-43f6-aa8b-0863dd781673'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:22.741918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-69b62e98-4e87-43f6-aa8b-0863dd781673 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc902dcd5bcd47c1bb3f4e87496fcec0', 'request_id': 'req-69b62e98-4e87-43f6-aa8b-0863dd781673'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:22.753211 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-69b62e98-4e87-43f6-aa8b-0863dd781673 admin admin] The token cache is being invalidate because project dc902dcd5bcd47c1bb3f4e87496fcec0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:36:22.753496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-69b62e98-4e87-43f6-aa8b-0863dd781673 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:22.754110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-69b62e98-4e87-43f6-aa8b-0863dd781673 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:22.755692 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 965/1919] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:36:22 2022] DELETE /identity/v3/projects/dc902dcd5bcd47c1bb3f4e87496fcec0 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:22.761887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8bfa18-fa07-4c41-96fa-0cf75b816b06 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:22.762369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8bfa18-fa07-4c41-96fa-0cf75b816b06 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:22.762460 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8bfa18-fa07-4c41-96fa-0cf75b816b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:22.783553 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-8f8bfa18-fa07-4c41-96fa-0cf75b816b06 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:22.784787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-8f8bfa18-fa07-4c41-96fa-0cf75b816b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:22.804670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:22.848084 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 955/1920] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:22.857102 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b9881-9042-4608-9eab-e47ef4fde59e None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:22.857390 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b9881-9042-4608-9eab-e47ef4fde59e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:22.857843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b9881-9042-4608-9eab-e47ef4fde59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:22.879331 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-a85b9881-9042-4608-9eab-e47ef4fde59e None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:22.880773 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a85b9881-9042-4608-9eab-e47ef4fde59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:22.897171 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 966/1921] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:22.901761 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8bfa18-fa07-4c41-96fa-0cf75b816b06 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:22.917307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8bfa18-fa07-4c41-96fa-0cf75b816b06 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:22.926199 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063bbd58-0d9f-4333-b844-023fc6f22f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:22.926199 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bbd58-0d9f-4333-b844-023fc6f22f5a admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:22.926199 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bbd58-0d9f-4333-b844-023fc6f22f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:22.926199 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bbd58-0d9f-4333-b844-023fc6f22f5a admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:22.927215 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063bbd58-0d9f-4333-b844-023fc6f22f5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:36:22.927693 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063bbd58-0d9f-4333-b844-023fc6f22f5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:22.929903 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063bbd58-0d9f-4333-b844-023fc6f22f5a admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:22.943718 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 956/1922] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:36:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:36:22.947252 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b9881-9042-4608-9eab-e47ef4fde59e None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:22.957723 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b9881-9042-4608-9eab-e47ef4fde59e None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:22.962611 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460e35e-2366-4503-b19a-69d6208695e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:22.963424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460e35e-2366-4503-b19a-69d6208695e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:22.963713 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460e35e-2366-4503-b19a-69d6208695e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:22.964018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460e35e-2366-4503-b19a-69d6208695e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:22.964722 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b460e35e-2366-4503-b19a-69d6208695e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:22.966740 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b460e35e-2366-4503-b19a-69d6208695e5 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:22.976898 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:23.035080 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 967/1923] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:36:22 2022] POST /identity/v3/projects => generated 367 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:36:23.035995 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063bbd58-0d9f-4333-b844-023fc6f22f5a admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:23.047129 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063bbd58-0d9f-4333-b844-023fc6f22f5a admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:23.108400 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ae58f-4913-4f4c-9766-3a1d9165ed10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:23.109079 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ae58f-4913-4f4c-9766-3a1d9165ed10 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:23.109385 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ae58f-4913-4f4c-9766-3a1d9165ed10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:23.109660 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ae58f-4913-4f4c-9766-3a1d9165ed10 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:23.110383 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ae58f-4913-4f4c-9766-3a1d9165ed10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:23.112406 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ae58f-4913-4f4c-9766-3a1d9165ed10 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:23.223223 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460e35e-2366-4503-b19a-69d6208695e5 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:23.225363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b460e35e-2366-4503-b19a-69d6208695e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:23.232262 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 957/1924] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:36:23 2022] POST /identity/v3/users => generated 284 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:36:23.236133 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ae58f-4913-4f4c-9766-3a1d9165ed10 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:23.243539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460e35e-2366-4503-b19a-69d6208695e5 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:23.246237 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ae58f-4913-4f4c-9766-3a1d9165ed10 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:23.248708 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5cd98-19a9-4a37-8820-fbd1e290c3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:23.249714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5cd98-19a9-4a37-8820-fbd1e290c3a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:23.250050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5cd98-19a9-4a37-8820-fbd1e290c3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:23.250321 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5cd98-19a9-4a37-8820-fbd1e290c3a3 admin admin] PATH_INFO: `/v3/users/f63d65fd793c42098ab4ad31643f37ac` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:23.251114 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbaaf25-c69e-4ad8-9d92-f436f3abe15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:23.251578 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc5cd98-19a9-4a37-8820-fbd1e290c3a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f63d65fd793c42098ab4ad31643f37ac)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:23.257050 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbaaf25-c69e-4ad8-9d92-f436f3abe15d admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:23.257050 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbaaf25-c69e-4ad8-9d92-f436f3abe15d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:23.257050 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbaaf25-c69e-4ad8-9d92-f436f3abe15d admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:23.257050 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbaaf25-c69e-4ad8-9d92-f436f3abe15d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:23.257362 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc5cd98-19a9-4a37-8820-fbd1e290c3a3 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:23.257444 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbaaf25-c69e-4ad8-9d92-f436f3abe15d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:23.265287 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 958/1925] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:36:23.269168 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbaaf25-c69e-4ad8-9d92-f436f3abe15d admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:23.279300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbaaf25-c69e-4ad8-9d92-f436f3abe15d admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:23.284022 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eab817-f75e-4f3b-ae38-5f7453d9be9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:23.290795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eab817-f75e-4f3b-ae38-5f7453d9be9f admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:23.290795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eab817-f75e-4f3b-ae38-5f7453d9be9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:23.290795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eab817-f75e-4f3b-ae38-5f7453d9be9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:23.290795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eab817-f75e-4f3b-ae38-5f7453d9be9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:23.290795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eab817-f75e-4f3b-ae38-5f7453d9be9f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:23.297956 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 959/1926] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:36:23.301849 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eab817-f75e-4f3b-ae38-5f7453d9be9f admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:23.311971 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eab817-f75e-4f3b-ae38-5f7453d9be9f admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:23.316621 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac4f11c-5d86-4e12-9d3e-c3f70565d377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:23.317460 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4f11c-5d86-4e12-9d3e-c3f70565d377 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:23.317743 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4f11c-5d86-4e12-9d3e-c3f70565d377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:23.318022 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4f11c-5d86-4e12-9d3e-c3f70565d377 admin admin] PATH_INFO: `/v3/projects/20fa1221b15f438baf91f6b6293e27c8/users/d03330e6199244449437db7d3d99ae8f/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:23.333205 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac4f11c-5d86-4e12-9d3e-c3f70565d377 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20fa1221b15f438baf91f6b6293e27c8, user_id=d03330e6199244449437db7d3d99ae8f, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:23.336604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac4f11c-5d86-4e12-9d3e-c3f70565d377 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:23.434667 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4bc5cd98-19a9-4a37-8820-fbd1e290c3a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f63d65fd793c42098ab4ad31643f37ac'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:23.459371 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 960/1927] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:36:23 2022] PUT /identity/v3/projects/20fa1221b15f438baf91f6b6293e27c8/users/d03330e6199244449437db7d3d99ae8f/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:36:23.471799 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e83761-6001-44e1-ba7f-ca916d791193 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:23.472105 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e83761-6001-44e1-ba7f-ca916d791193 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:23.472379 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e83761-6001-44e1-ba7f-ca916d791193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:23.502784 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e4e83761-6001-44e1-ba7f-ca916d791193 None None] MFA Rules not processed for user `d03330e6199244449437db7d3d99ae8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:23.503046 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e4e83761-6001-44e1-ba7f-ca916d791193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:23.524464 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4bc5cd98-19a9-4a37-8820-fbd1e290c3a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f63d65fd793c42098ab4ad31643f37ac'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:23.537822 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4bc5cd98-19a9-4a37-8820-fbd1e290c3a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f63d65fd793c42098ab4ad31643f37ac'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:23.539189 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4bc5cd98-19a9-4a37-8820-fbd1e290c3a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f63d65fd793c42098ab4ad31643f37ac'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:23.546292 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4bc5cd98-19a9-4a37-8820-fbd1e290c3a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f63d65fd793c42098ab4ad31643f37ac'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:23.552378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 968/1928] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:36:23 2022] DELETE /identity/v3/users/f63d65fd793c42098ab4ad31643f37ac => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:36:23.564907 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:23.586327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 961/1929] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:23 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:36:23.597720 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5cd98-19a9-4a37-8820-fbd1e290c3a3 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:23.602934 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4bc5cd98-19a9-4a37-8820-fbd1e290c3a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:23.665477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5cd98-19a9-4a37-8820-fbd1e290c3a3 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:23.672308 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445e60b-f795-4e72-a030-171422d832f7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95452d1b892a463d8e9e3eb16ca7c5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:23.672967 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445e60b-f795-4e72-a030-171422d832f7 service glance] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:23.673249 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445e60b-f795-4e72-a030-171422d832f7 service glance] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:23.673527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445e60b-f795-4e72-a030-171422d832f7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:23.734141 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f445e60b-f795-4e72-a030-171422d832f7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:23.735623 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f445e60b-f795-4e72-a030-171422d832f7 service glance] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:23.745302 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 969/1930] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:36:23 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:23.840457 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e83761-6001-44e1-ba7f-ca916d791193 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:23.842092 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e4e83761-6001-44e1-ba7f-ca916d791193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:23.925170 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e83761-6001-44e1-ba7f-ca916d791193 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:23.992080 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75192f0-2a6a-4ef8-8027-6dfd365764da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:23.992979 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75192f0-2a6a-4ef8-8027-6dfd365764da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:23.993254 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75192f0-2a6a-4ef8-8027-6dfd365764da admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:23.993525 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75192f0-2a6a-4ef8-8027-6dfd365764da admin admin] PATH_INFO: `/v3/projects/871210390ac4421a84321bef42e486c6` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:23.994827 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75192f0-2a6a-4ef8-8027-6dfd365764da admin admin] RBAC: Authorizing `identity:delete_project(project_id=871210390ac4421a84321bef42e486c6)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:23.996793 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75192f0-2a6a-4ef8-8027-6dfd365764da admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:24.164466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-f75192f0-2a6a-4ef8-8027-6dfd365764da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '871210390ac4421a84321bef42e486c6', 'request_id': 'req-f75192f0-2a6a-4ef8-8027-6dfd365764da'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:24.281302 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-f75192f0-2a6a-4ef8-8027-6dfd365764da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '871210390ac4421a84321bef42e486c6', 'request_id': 'req-f75192f0-2a6a-4ef8-8027-6dfd365764da'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:24.288311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-f75192f0-2a6a-4ef8-8027-6dfd365764da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '871210390ac4421a84321bef42e486c6', 'request_id': 'req-f75192f0-2a6a-4ef8-8027-6dfd365764da'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:24.299440 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-f75192f0-2a6a-4ef8-8027-6dfd365764da admin admin] The token cache is being invalidate because project 871210390ac4421a84321bef42e486c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:36:24.299440 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-f75192f0-2a6a-4ef8-8027-6dfd365764da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:24.300131 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-f75192f0-2a6a-4ef8-8027-6dfd365764da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:24.301679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 962/1931] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:36:23 2022] DELETE /identity/v3/projects/871210390ac4421a84321bef42e486c6 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:24.306141 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445e60b-f795-4e72-a030-171422d832f7 service glance] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:24.317856 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445e60b-f795-4e72-a030-171422d832f7 service glance] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:24.330214 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:24.374206 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a832b8a-af19-4abc-81f5-507e3894f039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:24.375118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a832b8a-af19-4abc-81f5-507e3894f039 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:24.375393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a832b8a-af19-4abc-81f5-507e3894f039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:24.375660 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a832b8a-af19-4abc-81f5-507e3894f039 admin admin] PATH_INFO: `/v3/users/5996958b85d842c4a340831a6107c01c` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:24.376822 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a832b8a-af19-4abc-81f5-507e3894f039 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5996958b85d842c4a340831a6107c01c)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:24.378767 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a832b8a-af19-4abc-81f5-507e3894f039 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:24.611660 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3a832b8a-af19-4abc-81f5-507e3894f039 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5996958b85d842c4a340831a6107c01c'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:24.724197 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3a832b8a-af19-4abc-81f5-507e3894f039 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5996958b85d842c4a340831a6107c01c'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:24.743699 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3a832b8a-af19-4abc-81f5-507e3894f039 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5996958b85d842c4a340831a6107c01c'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:24.745118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3a832b8a-af19-4abc-81f5-507e3894f039 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5996958b85d842c4a340831a6107c01c'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:24.753849 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3a832b8a-af19-4abc-81f5-507e3894f039 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5996958b85d842c4a340831a6107c01c'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:24.761633 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 970/1932] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:36:24 2022] DELETE /identity/v3/users/5996958b85d842c4a340831a6107c01c => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:36:25.111329 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75192f0-2a6a-4ef8-8027-6dfd365764da admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:25.112961 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f75192f0-2a6a-4ef8-8027-6dfd365764da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:25.129295 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75192f0-2a6a-4ef8-8027-6dfd365764da admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:25.150014 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:25.192384 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab71c76-1506-4d58-b7fa-a8e93971af23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:25.193245 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab71c76-1506-4d58-b7fa-a8e93971af23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:25.193522 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab71c76-1506-4d58-b7fa-a8e93971af23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:25.194528 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab71c76-1506-4d58-b7fa-a8e93971af23 admin admin] PATH_INFO: `/v3/projects/c58019bd2d474edfb227d08c601d453d` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:25.195896 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab71c76-1506-4d58-b7fa-a8e93971af23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c58019bd2d474edfb227d08c601d453d)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:25.197867 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab71c76-1506-4d58-b7fa-a8e93971af23 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:25.400953 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6ab71c76-1506-4d58-b7fa-a8e93971af23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c58019bd2d474edfb227d08c601d453d', 'request_id': 'req-6ab71c76-1506-4d58-b7fa-a8e93971af23'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:25.541833 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6ab71c76-1506-4d58-b7fa-a8e93971af23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c58019bd2d474edfb227d08c601d453d', 'request_id': 'req-6ab71c76-1506-4d58-b7fa-a8e93971af23'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:25.550666 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6ab71c76-1506-4d58-b7fa-a8e93971af23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c58019bd2d474edfb227d08c601d453d', 'request_id': 'req-6ab71c76-1506-4d58-b7fa-a8e93971af23'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:25.563311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6ab71c76-1506-4d58-b7fa-a8e93971af23 admin admin] The token cache is being invalidate because project c58019bd2d474edfb227d08c601d453d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:36:25.563599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6ab71c76-1506-4d58-b7fa-a8e93971af23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:25.564354 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6ab71c76-1506-4d58-b7fa-a8e93971af23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:25.565943 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 963/1933] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:36:25 2022] DELETE /identity/v3/projects/c58019bd2d474edfb227d08c601d453d => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:25.571676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae01c5-54bb-4012-95dd-eed1ea67ab31 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:25.571963 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae01c5-54bb-4012-95dd-eed1ea67ab31 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:25.572293 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ae01c5-54bb-4012-95dd-eed1ea67ab31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:25.595286 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-12ae01c5-54bb-4012-95dd-eed1ea67ab31 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:25.596590 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-12ae01c5-54bb-4012-95dd-eed1ea67ab31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:25.662478 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 971/1934] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:25.670859 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2700b9-1962-4539-9e59-23b0a5473a29 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:25.671166 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2700b9-1962-4539-9e59-23b0a5473a29 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:25.671446 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2700b9-1962-4539-9e59-23b0a5473a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:25.712514 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-8f2700b9-1962-4539-9e59-23b0a5473a29 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:25.713863 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-8f2700b9-1962-4539-9e59-23b0a5473a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:25.729159 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 964/1935] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:25.734749 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae01c5-54bb-4012-95dd-eed1ea67ab31 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:25.744382 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ae01c5-54bb-4012-95dd-eed1ea67ab31 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:25.749980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0e543-70e0-4340-853d-5d5648c07d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:25.751103 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0e543-70e0-4340-853d-5d5648c07d7e admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:25.751588 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0e543-70e0-4340-853d-5d5648c07d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:25.752045 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0e543-70e0-4340-853d-5d5648c07d7e admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:25.753007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0e543-70e0-4340-853d-5d5648c07d7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:36:25.753379 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0e543-70e0-4340-853d-5d5648c07d7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:25.755276 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0e543-70e0-4340-853d-5d5648c07d7e admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:25.775398 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 972/1936] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:36:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:36:25.776882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2700b9-1962-4539-9e59-23b0a5473a29 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:25.795268 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2700b9-1962-4539-9e59-23b0a5473a29 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:25.799173 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5331d-7e58-49aa-9256-3ed1aa85deb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:25.800306 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5331d-7e58-49aa-9256-3ed1aa85deb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:25.805031 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5331d-7e58-49aa-9256-3ed1aa85deb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:25.805031 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5331d-7e58-49aa-9256-3ed1aa85deb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:25.805031 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c5331d-7e58-49aa-9256-3ed1aa85deb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:25.805031 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c5331d-7e58-49aa-9256-3ed1aa85deb7 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:25.941234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 965/1937] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:36:25 2022] POST /identity/v3/projects => generated 385 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:36:25.946260 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0e543-70e0-4340-853d-5d5648c07d7e admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:25.962119 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0e543-70e0-4340-853d-5d5648c07d7e admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:26.011026 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:26.052039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ff9af-551d-4594-846d-d7274eb2e7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:26.052640 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ff9af-551d-4594-846d-d7274eb2e7ac admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:26.052926 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ff9af-551d-4594-846d-d7274eb2e7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:26.053235 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ff9af-551d-4594-846d-d7274eb2e7ac admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:26.054569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570ff9af-551d-4594-846d-d7274eb2e7ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:26.056618 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570ff9af-551d-4594-846d-d7274eb2e7ac admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:26.211148 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 973/1938] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:36:25 2022] POST /identity/v3/users => generated 293 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:36:26.215189 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5331d-7e58-49aa-9256-3ed1aa85deb7 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:26.225803 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5331d-7e58-49aa-9256-3ed1aa85deb7 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:26.230362 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3fb0c-d4c2-41e4-97f3-5a6b5547a181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:26.231069 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3fb0c-d4c2-41e4-97f3-5a6b5547a181 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:26.231383 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3fb0c-d4c2-41e4-97f3-5a6b5547a181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:26.231637 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3fb0c-d4c2-41e4-97f3-5a6b5547a181 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:26.232358 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3fb0c-d4c2-41e4-97f3-5a6b5547a181 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:26.233968 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3fb0c-d4c2-41e4-97f3-5a6b5547a181 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:26.245565 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 966/1939] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:26.250632 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ff9af-551d-4594-846d-d7274eb2e7ac admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:26.260802 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ff9af-551d-4594-846d-d7274eb2e7ac admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:26.265561 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ccd0a2-be1a-4378-bf22-ea0bb2a4a722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:26.267381 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ccd0a2-be1a-4378-bf22-ea0bb2a4a722 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:26.267671 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ccd0a2-be1a-4378-bf22-ea0bb2a4a722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:26.267953 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ccd0a2-be1a-4378-bf22-ea0bb2a4a722 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:26.268680 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ccd0a2-be1a-4378-bf22-ea0bb2a4a722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:26.270286 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ccd0a2-be1a-4378-bf22-ea0bb2a4a722 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:26.280846 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 974/1940] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:26.285017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3fb0c-d4c2-41e4-97f3-5a6b5547a181 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:26.295797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3fb0c-d4c2-41e4-97f3-5a6b5547a181 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:26.300692 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a1804-de27-42db-908a-7f2082a8cadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:26.301220 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a1804-de27-42db-908a-7f2082a8cadc admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:26.301496 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a1804-de27-42db-908a-7f2082a8cadc admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:26.301780 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74a1804-de27-42db-908a-7f2082a8cadc admin admin] PATH_INFO: `/v3/projects/23520db827e14bff85e92764b7d6a2bc/users/88e2b9522ee44379a76a7d1a8d2b32d0/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:26.316480 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a1804-de27-42db-908a-7f2082a8cadc admin admin] RBAC: Authorizing `identity:create_grant(project_id=23520db827e14bff85e92764b7d6a2bc, user_id=88e2b9522ee44379a76a7d1a8d2b32d0, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:26.319264 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74a1804-de27-42db-908a-7f2082a8cadc admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:26.404558 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 967/1941] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:36:26 2022] PUT /identity/v3/projects/23520db827e14bff85e92764b7d6a2bc/users/88e2b9522ee44379a76a7d1a8d2b32d0/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:26.416340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c3657-10d7-40a7-91f2-dd3a5b41f259 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:26.416632 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c3657-10d7-40a7-91f2-dd3a5b41f259 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:26.416913 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c3657-10d7-40a7-91f2-dd3a5b41f259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:26.443175 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-2d9c3657-10d7-40a7-91f2-dd3a5b41f259 None None] MFA Rules not processed for user `88e2b9522ee44379a76a7d1a8d2b32d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:26.444421 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-2d9c3657-10d7-40a7-91f2-dd3a5b41f259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:26.519804 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 975/1942] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:26.531519 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a1804-de27-42db-908a-7f2082a8cadc admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:26.533214 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d74a1804-de27-42db-908a-7f2082a8cadc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:26.587774 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:26.592859 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74a1804-de27-42db-908a-7f2082a8cadc admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:26.600156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc4d26-c4ca-46ba-bd3e-bbddd0f70ffd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:26.601452 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc4d26-c4ca-46ba-bd3e-bbddd0f70ffd service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:26.601744 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc4d26-c4ca-46ba-bd3e-bbddd0f70ffd service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:26.602014 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc4d26-c4ca-46ba-bd3e-bbddd0f70ffd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:26.610669 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc4d26-c4ca-46ba-bd3e-bbddd0f70ffd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:26.612188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc4d26-c4ca-46ba-bd3e-bbddd0f70ffd service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:26.621787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 968/1943] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:36:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:26.643480 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c3657-10d7-40a7-91f2-dd3a5b41f259 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:26.714296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c3657-10d7-40a7-91f2-dd3a5b41f259 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:26.719014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc223eaf-748f-40f8-b8ba-8523f612a002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:26.719879 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc223eaf-748f-40f8-b8ba-8523f612a002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:26.720156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc223eaf-748f-40f8-b8ba-8523f612a002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:26.720425 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc223eaf-748f-40f8-b8ba-8523f612a002 admin admin] PATH_INFO: `/v3/users/88e2b9522ee44379a76a7d1a8d2b32d0` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:26.721611 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc223eaf-748f-40f8-b8ba-8523f612a002 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88e2b9522ee44379a76a7d1a8d2b32d0)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:26.723592 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc223eaf-748f-40f8-b8ba-8523f612a002 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:26.917904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:26.930864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-fc223eaf-748f-40f8-b8ba-8523f612a002 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88e2b9522ee44379a76a7d1a8d2b32d0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:27.001525 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-fc223eaf-748f-40f8-b8ba-8523f612a002 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88e2b9522ee44379a76a7d1a8d2b32d0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:27.016935 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-fc223eaf-748f-40f8-b8ba-8523f612a002 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88e2b9522ee44379a76a7d1a8d2b32d0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:27.017957 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-fc223eaf-748f-40f8-b8ba-8523f612a002 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88e2b9522ee44379a76a7d1a8d2b32d0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:27.032611 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-fc223eaf-748f-40f8-b8ba-8523f612a002 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88e2b9522ee44379a76a7d1a8d2b32d0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:27.035826 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc4d26-c4ca-46ba-bd3e-bbddd0f70ffd service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:27.037531 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-22bc4d26-c4ca-46ba-bd3e-bbddd0f70ffd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:27.042090 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 976/1944] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:36:26 2022] DELETE /identity/v3/users/88e2b9522ee44379a76a7d1a8d2b32d0 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:27.043754 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87962a86-7a08-4686-bed1-4f1ecbbbc975 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:27.044340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87962a86-7a08-4686-bed1-4f1ecbbbc975 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:27.045050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87962a86-7a08-4686-bed1-4f1ecbbbc975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:27.078500 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc4d26-c4ca-46ba-bd3e-bbddd0f70ffd service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:27.084003 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-87962a86-7a08-4686-bed1-4f1ecbbbc975 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:27.086366 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-87962a86-7a08-4686-bed1-4f1ecbbbc975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:27.162461 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926ef3a-db3c-42a8-8c6b-f8e189cf74ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:27.164281 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926ef3a-db3c-42a8-8c6b-f8e189cf74ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:27.164778 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926ef3a-db3c-42a8-8c6b-f8e189cf74ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:27.165346 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926ef3a-db3c-42a8-8c6b-f8e189cf74ad admin admin] PATH_INFO: `/v3/users/4eb23449930743058cca3969a20a3b51` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:27.167132 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8926ef3a-db3c-42a8-8c6b-f8e189cf74ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=4eb23449930743058cca3969a20a3b51)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:27.175149 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8926ef3a-db3c-42a8-8c6b-f8e189cf74ad admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:27.183195 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 977/1945] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:27.192689 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87962a86-7a08-4686-bed1-4f1ecbbbc975 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:27.194943 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-87962a86-7a08-4686-bed1-4f1ecbbbc975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:27.300309 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87962a86-7a08-4686-bed1-4f1ecbbbc975 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:27.305236 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f3ae1-1953-4a2f-b07b-741ccd5be9d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:27.306125 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f3ae1-1953-4a2f-b07b-741ccd5be9d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:27.306457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f3ae1-1953-4a2f-b07b-741ccd5be9d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:27.306737 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f3ae1-1953-4a2f-b07b-741ccd5be9d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:27.316492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f3ae1-1953-4a2f-b07b-741ccd5be9d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:27.318220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f3ae1-1953-4a2f-b07b-741ccd5be9d8 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:27.333557 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 978/1946] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:36:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:36:27.402766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-8926ef3a-db3c-42a8-8c6b-f8e189cf74ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4eb23449930743058cca3969a20a3b51'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:27.481373 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-8926ef3a-db3c-42a8-8c6b-f8e189cf74ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4eb23449930743058cca3969a20a3b51'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:27.482079 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-8926ef3a-db3c-42a8-8c6b-f8e189cf74ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4eb23449930743058cca3969a20a3b51'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:27.496609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-8926ef3a-db3c-42a8-8c6b-f8e189cf74ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4eb23449930743058cca3969a20a3b51'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:27.497249 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-8926ef3a-db3c-42a8-8c6b-f8e189cf74ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4eb23449930743058cca3969a20a3b51'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:27.525152 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 969/1947] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:36:27 2022] DELETE /identity/v3/users/4eb23449930743058cca3969a20a3b51 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:27.530396 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1b52d-7a48-408b-8895-49810cbdb4d9 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:27.530664 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1b52d-7a48-408b-8895-49810cbdb4d9 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:27.531070 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1b52d-7a48-408b-8895-49810cbdb4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:27.549432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-71f1b52d-7a48-408b-8895-49810cbdb4d9 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:27.550861 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-71f1b52d-7a48-408b-8895-49810cbdb4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:27.611669 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:27.625839 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 979/1948] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:27.637036 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926ef3a-db3c-42a8-8c6b-f8e189cf74ad admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:27.638650 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-8926ef3a-db3c-42a8-8c6b-f8e189cf74ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:27.701120 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926ef3a-db3c-42a8-8c6b-f8e189cf74ad admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:27.710745 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb1d98-dfcf-439b-bc51-eb6d19bc420d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:27.711590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb1d98-dfcf-439b-bc51-eb6d19bc420d service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:27.712081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb1d98-dfcf-439b-bc51-eb6d19bc420d service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:27.712349 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb1d98-dfcf-439b-bc51-eb6d19bc420d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:27.727306 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb1d98-dfcf-439b-bc51-eb6d19bc420d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:27.727399 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb1d98-dfcf-439b-bc51-eb6d19bc420d service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:27.738589 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 970/1949] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:36:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:27.795577 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1b52d-7a48-408b-8895-49810cbdb4d9 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:27.797082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-71f1b52d-7a48-408b-8895-49810cbdb4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:27.809950 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1b52d-7a48-408b-8895-49810cbdb4d9 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:27.815485 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd72185-ee91-416d-b9fc-4656a86e3e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:27.819522 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd72185-ee91-416d-b9fc-4656a86e3e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:27.819522 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd72185-ee91-416d-b9fc-4656a86e3e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:27.819522 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd72185-ee91-416d-b9fc-4656a86e3e01 admin admin] PATH_INFO: `/v3/projects/23520db827e14bff85e92764b7d6a2bc` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:27.819522 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd72185-ee91-416d-b9fc-4656a86e3e01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23520db827e14bff85e92764b7d6a2bc)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:27.820097 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd72185-ee91-416d-b9fc-4656a86e3e01 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:27.969480 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-edd72185-ee91-416d-b9fc-4656a86e3e01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23520db827e14bff85e92764b7d6a2bc', 'request_id': 'req-edd72185-ee91-416d-b9fc-4656a86e3e01'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:28.061807 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-edd72185-ee91-416d-b9fc-4656a86e3e01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23520db827e14bff85e92764b7d6a2bc', 'request_id': 'req-edd72185-ee91-416d-b9fc-4656a86e3e01'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:28.068646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-edd72185-ee91-416d-b9fc-4656a86e3e01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23520db827e14bff85e92764b7d6a2bc', 'request_id': 'req-edd72185-ee91-416d-b9fc-4656a86e3e01'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:28.088572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-edd72185-ee91-416d-b9fc-4656a86e3e01 admin admin] The token cache is being invalidate because project 23520db827e14bff85e92764b7d6a2bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:36:28.089440 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-edd72185-ee91-416d-b9fc-4656a86e3e01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:28.089714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-edd72185-ee91-416d-b9fc-4656a86e3e01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:28.091398 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 980/1950] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:36:27 2022] DELETE /identity/v3/projects/23520db827e14bff85e92764b7d6a2bc => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:28.096481 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da98661-22bb-4c83-9a8e-91888c1777ce None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:28.096822 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da98661-22bb-4c83-9a8e-91888c1777ce None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:28.097109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da98661-22bb-4c83-9a8e-91888c1777ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:28.116549 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-9da98661-22bb-4c83-9a8e-91888c1777ce None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:28.117746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-9da98661-22bb-4c83-9a8e-91888c1777ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:28.123079 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:28.176433 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 971/1951] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:36:28.184935 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923888d-45cb-454b-94fd-ca60ca2d4658 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:28.185337 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923888d-45cb-454b-94fd-ca60ca2d4658 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:28.185714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1923888d-45cb-454b-94fd-ca60ca2d4658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:28.194480 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da98661-22bb-4c83-9a8e-91888c1777ce None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:28.196066 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-9da98661-22bb-4c83-9a8e-91888c1777ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:28.208520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da98661-22bb-4c83-9a8e-91888c1777ce None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:28.212932 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-1923888d-45cb-454b-94fd-ca60ca2d4658 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:28.213353 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047960a7-531b-4398-869c-9efef75c33da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:28.214172 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1923888d-45cb-454b-94fd-ca60ca2d4658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:28.214313 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047960a7-531b-4398-869c-9efef75c33da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:28.214496 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047960a7-531b-4398-869c-9efef75c33da admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:28.214774 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047960a7-531b-4398-869c-9efef75c33da admin admin] PATH_INFO: `/v3/projects/625821c00c8d4c9f843e86386e4c61e5` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:28.215987 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047960a7-531b-4398-869c-9efef75c33da admin admin] RBAC: Authorizing `identity:delete_project(project_id=625821c00c8d4c9f843e86386e4c61e5)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:28.217991 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047960a7-531b-4398-869c-9efef75c33da admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:28.230742 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 981/1952] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:28.235005 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923888d-45cb-454b-94fd-ca60ca2d4658 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:28.244571 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1923888d-45cb-454b-94fd-ca60ca2d4658 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:28.249510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe135b89-f702-4006-818e-cb9a884869d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:28.250214 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe135b89-f702-4006-818e-cb9a884869d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:28.250491 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe135b89-f702-4006-818e-cb9a884869d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:28.250762 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe135b89-f702-4006-818e-cb9a884869d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:28.251600 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe135b89-f702-4006-818e-cb9a884869d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:36:28.252019 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe135b89-f702-4006-818e-cb9a884869d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:28.253644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe135b89-f702-4006-818e-cb9a884869d9 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:28.265398 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 982/1953] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:36:28.269879 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe135b89-f702-4006-818e-cb9a884869d9 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:28.279963 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe135b89-f702-4006-818e-cb9a884869d9 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:28.284620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a5223-d366-41f6-a480-1a5e0d0fe7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:28.287025 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a5223-d366-41f6-a480-1a5e0d0fe7ce admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:28.287025 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a5223-d366-41f6-a480-1a5e0d0fe7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:28.287025 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a5223-d366-41f6-a480-1a5e0d0fe7ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:28.287025 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a5223-d366-41f6-a480-1a5e0d0fe7ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:28.288920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a5223-d366-41f6-a480-1a5e0d0fe7ce admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:28.322899 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-047960a7-531b-4398-869c-9efef75c33da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '625821c00c8d4c9f843e86386e4c61e5', 'request_id': 'req-047960a7-531b-4398-869c-9efef75c33da'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:28.405314 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 983/1954] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:36:28 2022] POST /identity/v3/projects => generated 375 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:36:28.407960 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-047960a7-531b-4398-869c-9efef75c33da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '625821c00c8d4c9f843e86386e4c61e5', 'request_id': 'req-047960a7-531b-4398-869c-9efef75c33da'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:28.409615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a5223-d366-41f6-a480-1a5e0d0fe7ce admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:28.415487 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-047960a7-531b-4398-869c-9efef75c33da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '625821c00c8d4c9f843e86386e4c61e5', 'request_id': 'req-047960a7-531b-4398-869c-9efef75c33da'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:28.425910 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-047960a7-531b-4398-869c-9efef75c33da admin admin] The token cache is being invalidate because project 625821c00c8d4c9f843e86386e4c61e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:36:28.426180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-047960a7-531b-4398-869c-9efef75c33da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:28.426894 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-047960a7-531b-4398-869c-9efef75c33da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:28.429109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 972/1955] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:36:28 2022] DELETE /identity/v3/projects/625821c00c8d4c9f843e86386e4c61e5 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:36:28.436757 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047960a7-531b-4398-869c-9efef75c33da admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:28.438262 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-047960a7-531b-4398-869c-9efef75c33da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:28.472890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a5223-d366-41f6-a480-1a5e0d0fe7ce admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:28.478426 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba7429-3238-4ec9-828f-2c5409b6ee7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:28.479123 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba7429-3238-4ec9-828f-2c5409b6ee7d admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:28.479402 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba7429-3238-4ec9-828f-2c5409b6ee7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:28.479939 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba7429-3238-4ec9-828f-2c5409b6ee7d admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:28.480448 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba7429-3238-4ec9-828f-2c5409b6ee7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:28.482877 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba7429-3238-4ec9-828f-2c5409b6ee7d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:28.502330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047960a7-531b-4398-869c-9efef75c33da admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:28.507072 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f74410-596a-4edc-b525-3de879bf0eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:28.507951 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f74410-596a-4edc-b525-3de879bf0eb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:28.508232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f74410-596a-4edc-b525-3de879bf0eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:28.508512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f74410-596a-4edc-b525-3de879bf0eb2 admin admin] PATH_INFO: `/v3/users/4922c7269260456eb5d24d1388f1f995` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:28.509843 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f74410-596a-4edc-b525-3de879bf0eb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4922c7269260456eb5d24d1388f1f995)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:28.511811 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f74410-596a-4edc-b525-3de879bf0eb2 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:28.561434 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 984/1956] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:36:28 2022] POST /identity/v3/users => generated 288 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:36:28.565644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba7429-3238-4ec9-828f-2c5409b6ee7d admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:28.567305 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d8ba7429-3238-4ec9-828f-2c5409b6ee7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:28.579355 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba7429-3238-4ec9-828f-2c5409b6ee7d admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:28.583965 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f8777-8b6c-44ba-a571-0a4657e45333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:28.585105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f8777-8b6c-44ba-a571-0a4657e45333 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:28.585388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f8777-8b6c-44ba-a571-0a4657e45333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:28.585747 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040f8777-8b6c-44ba-a571-0a4657e45333 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:28.586509 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f8777-8b6c-44ba-a571-0a4657e45333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:28.588137 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040f8777-8b6c-44ba-a571-0a4657e45333 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:28.598051 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 985/1957] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:36:28.602351 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f8777-8b6c-44ba-a571-0a4657e45333 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:28.611784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040f8777-8b6c-44ba-a571-0a4657e45333 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:28.616416 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef2ff0-b9d2-4a97-86fc-ef9132cd485c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:28.617113 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef2ff0-b9d2-4a97-86fc-ef9132cd485c admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:28.617505 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef2ff0-b9d2-4a97-86fc-ef9132cd485c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:28.617784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef2ff0-b9d2-4a97-86fc-ef9132cd485c admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:28.618513 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef2ff0-b9d2-4a97-86fc-ef9132cd485c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:28.620366 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef2ff0-b9d2-4a97-86fc-ef9132cd485c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:28.630090 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 986/1958] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:36:28.634007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef2ff0-b9d2-4a97-86fc-ef9132cd485c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:28.638597 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:28.643504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef2ff0-b9d2-4a97-86fc-ef9132cd485c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:28.648646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2d412-7819-4538-af2e-ccc42f36d94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:28.649462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2d412-7819-4538-af2e-ccc42f36d94d admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:28.649743 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2d412-7819-4538-af2e-ccc42f36d94d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:28.650014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2d412-7819-4538-af2e-ccc42f36d94d admin admin] PATH_INFO: `/v3/projects/d069fd1d39354fc6a02b0f76de7dffe7/users/ba54577bec664442a1bd8d4ac3b9085b/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:28.664735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2d412-7819-4538-af2e-ccc42f36d94d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d069fd1d39354fc6a02b0f76de7dffe7, user_id=ba54577bec664442a1bd8d4ac3b9085b, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:28.667403 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2d412-7819-4538-af2e-ccc42f36d94d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:28.691096 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c6f74410-596a-4edc-b525-3de879bf0eb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4922c7269260456eb5d24d1388f1f995'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:28.706815 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:28.764841 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 987/1959] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:36:28 2022] PUT /identity/v3/projects/d069fd1d39354fc6a02b0f76de7dffe7/users/ba54577bec664442a1bd8d4ac3b9085b/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:36:28.776660 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853f3db-a4de-4b85-b54b-424f8ec19522 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:28.776954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853f3db-a4de-4b85-b54b-424f8ec19522 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:28.777216 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853f3db-a4de-4b85-b54b-424f8ec19522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:28.779918 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c6f74410-596a-4edc-b525-3de879bf0eb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4922c7269260456eb5d24d1388f1f995'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:28.780879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c6f74410-596a-4edc-b525-3de879bf0eb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4922c7269260456eb5d24d1388f1f995'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:28.792196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c6f74410-596a-4edc-b525-3de879bf0eb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4922c7269260456eb5d24d1388f1f995'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:28.792981 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c6f74410-596a-4edc-b525-3de879bf0eb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4922c7269260456eb5d24d1388f1f995'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:28.809453 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-6853f3db-a4de-4b85-b54b-424f8ec19522 None None] MFA Rules not processed for user `ba54577bec664442a1bd8d4ac3b9085b`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:28.810747 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6853f3db-a4de-4b85-b54b-424f8ec19522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:28.812249 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 973/1960] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:36:28 2022] DELETE /identity/v3/users/4922c7269260456eb5d24d1388f1f995 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:36:28.888870 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 988/1961] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:28.899974 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f74410-596a-4edc-b525-3de879bf0eb2 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:28.901388 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c6f74410-596a-4edc-b525-3de879bf0eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:28.959957 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f74410-596a-4edc-b525-3de879bf0eb2 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:28.965725 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070fbd3-4212-4d40-8d99-bc1b6ce19193 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95452d1b892a463d8e9e3eb16ca7c5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:28.967578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070fbd3-4212-4d40-8d99-bc1b6ce19193 service glance] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:28.969804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070fbd3-4212-4d40-8d99-bc1b6ce19193 service glance] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:28.969804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070fbd3-4212-4d40-8d99-bc1b6ce19193 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:28.980264 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070fbd3-4212-4d40-8d99-bc1b6ce19193 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:28.981857 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8070fbd3-4212-4d40-8d99-bc1b6ce19193 service glance] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:28.998735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 974/1962] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:36:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:29.305471 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853f3db-a4de-4b85-b54b-424f8ec19522 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:29.306337 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6853f3db-a4de-4b85-b54b-424f8ec19522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:29.374432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6853f3db-a4de-4b85-b54b-424f8ec19522 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:29.379036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7cacbc-e720-4859-9dfc-fedf55a78a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:29.379906 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7cacbc-e720-4859-9dfc-fedf55a78a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:29.380186 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7cacbc-e720-4859-9dfc-fedf55a78a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:29.380457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7cacbc-e720-4859-9dfc-fedf55a78a9e admin admin] PATH_INFO: `/v3/projects/1bf7812778274edea98c8e09cd5522dd` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:29.381666 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7cacbc-e720-4859-9dfc-fedf55a78a9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bf7812778274edea98c8e09cd5522dd)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:29.383622 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7cacbc-e720-4859-9dfc-fedf55a78a9e admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:29.529993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-2e7cacbc-e720-4859-9dfc-fedf55a78a9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bf7812778274edea98c8e09cd5522dd', 'request_id': 'req-2e7cacbc-e720-4859-9dfc-fedf55a78a9e'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:29.592321 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-2e7cacbc-e720-4859-9dfc-fedf55a78a9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bf7812778274edea98c8e09cd5522dd', 'request_id': 'req-2e7cacbc-e720-4859-9dfc-fedf55a78a9e'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:29.598287 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-2e7cacbc-e720-4859-9dfc-fedf55a78a9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bf7812778274edea98c8e09cd5522dd', 'request_id': 'req-2e7cacbc-e720-4859-9dfc-fedf55a78a9e'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:29.605923 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:29.610483 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-2e7cacbc-e720-4859-9dfc-fedf55a78a9e admin admin] The token cache is being invalidate because project 1bf7812778274edea98c8e09cd5522dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:36:29.610768 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-2e7cacbc-e720-4859-9dfc-fedf55a78a9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:29.611435 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-2e7cacbc-e720-4859-9dfc-fedf55a78a9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:29.612977 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 989/1963] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:36:29 2022] DELETE /identity/v3/projects/1bf7812778274edea98c8e09cd5522dd => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:29.618960 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c031e5-ea8d-4fc3-bca8-637f07e00214 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:29.619445 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c031e5-ea8d-4fc3-bca8-637f07e00214 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:29.619645 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c031e5-ea8d-4fc3-bca8-637f07e00214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:29.637643 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-49c031e5-ea8d-4fc3-bca8-637f07e00214 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:29.638818 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-49c031e5-ea8d-4fc3-bca8-637f07e00214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:29.697021 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 975/1964] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:29.711093 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b175112-92c4-48f5-baa5-49fcc51cf888 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:29.711384 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b175112-92c4-48f5-baa5-49fcc51cf888 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:29.711664 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b175112-92c4-48f5-baa5-49fcc51cf888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:29.730245 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-1b175112-92c4-48f5-baa5-49fcc51cf888 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:29.731494 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1b175112-92c4-48f5-baa5-49fcc51cf888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:29.739360 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 990/1965] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:29.743721 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c031e5-ea8d-4fc3-bca8-637f07e00214 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:29.754331 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c031e5-ea8d-4fc3-bca8-637f07e00214 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:29.761581 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad88e1-60e6-4d24-9614-59444be9f011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:29.762537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad88e1-60e6-4d24-9614-59444be9f011 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:29.762537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad88e1-60e6-4d24-9614-59444be9f011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:29.762923 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebad88e1-60e6-4d24-9614-59444be9f011 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:29.763668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebad88e1-60e6-4d24-9614-59444be9f011 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:36:29.763974 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebad88e1-60e6-4d24-9614-59444be9f011 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:29.765578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebad88e1-60e6-4d24-9614-59444be9f011 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:29.778171 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 976/1966] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:36:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:36:29.782528 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b175112-92c4-48f5-baa5-49fcc51cf888 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:29.793070 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b175112-92c4-48f5-baa5-49fcc51cf888 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:29.798581 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c475ac-729e-4a9c-9460-8bdeba793c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:29.799419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c475ac-729e-4a9c-9460-8bdeba793c61 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:29.799765 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c475ac-729e-4a9c-9460-8bdeba793c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:29.800106 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c475ac-729e-4a9c-9460-8bdeba793c61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:29.800882 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c475ac-729e-4a9c-9460-8bdeba793c61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:29.803001 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c475ac-729e-4a9c-9460-8bdeba793c61 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:29.941543 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 991/1967] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:36:29 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:36:29.945943 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad88e1-60e6-4d24-9614-59444be9f011 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:29.957294 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebad88e1-60e6-4d24-9614-59444be9f011 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:30.005010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:30.023892 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566145f1-4211-4921-9845-c6ffc1375792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:30.027844 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566145f1-4211-4921-9845-c6ffc1375792 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:30.027844 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566145f1-4211-4921-9845-c6ffc1375792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:30.027844 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566145f1-4211-4921-9845-c6ffc1375792 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:30.027844 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566145f1-4211-4921-9845-c6ffc1375792 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:30.028741 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566145f1-4211-4921-9845-c6ffc1375792 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:30.186536 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 977/1968] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:36:29 2022] POST /identity/v3/users => generated 287 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:36:30.190358 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c475ac-729e-4a9c-9460-8bdeba793c61 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:30.201385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c475ac-729e-4a9c-9460-8bdeba793c61 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:30.206074 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f9baf-ac10-44e3-b29b-ca3f110594da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:30.206739 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f9baf-ac10-44e3-b29b-ca3f110594da admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:30.207056 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f9baf-ac10-44e3-b29b-ca3f110594da admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:30.207354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f9baf-ac10-44e3-b29b-ca3f110594da admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:30.208058 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f9baf-ac10-44e3-b29b-ca3f110594da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:30.209877 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f9baf-ac10-44e3-b29b-ca3f110594da admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:30.220402 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 992/1969] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:30.229546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566145f1-4211-4921-9845-c6ffc1375792 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:30.239768 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566145f1-4211-4921-9845-c6ffc1375792 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:30.244900 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ed5e6-dbcc-467f-bf05-67221de651b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:30.245554 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ed5e6-dbcc-467f-bf05-67221de651b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:30.245837 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ed5e6-dbcc-467f-bf05-67221de651b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:30.246107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ed5e6-dbcc-467f-bf05-67221de651b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:30.246827 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8ed5e6-dbcc-467f-bf05-67221de651b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:30.248445 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8ed5e6-dbcc-467f-bf05-67221de651b5 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:30.259498 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 978/1970] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:30.264502 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f9baf-ac10-44e3-b29b-ca3f110594da admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:30.275006 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f9baf-ac10-44e3-b29b-ca3f110594da admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:30.279555 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe506f3-b0d4-49db-ad96-5539899df533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:30.280482 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe506f3-b0d4-49db-ad96-5539899df533 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:30.280763 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe506f3-b0d4-49db-ad96-5539899df533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:30.281052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe506f3-b0d4-49db-ad96-5539899df533 admin admin] PATH_INFO: `/v3/projects/5c0f8424568846938ab720101b7cf794/users/dbea161f89174362ab1bd3231b582977/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:30.295843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe506f3-b0d4-49db-ad96-5539899df533 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c0f8424568846938ab720101b7cf794, user_id=dbea161f89174362ab1bd3231b582977, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:30.298681 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe506f3-b0d4-49db-ad96-5539899df533 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:30.372741 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 993/1971] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:36:30 2022] PUT /identity/v3/projects/5c0f8424568846938ab720101b7cf794/users/dbea161f89174362ab1bd3231b582977/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:30.385406 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa176b87-7bf4-4436-b7df-dd197be5ea46 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:30.385697 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa176b87-7bf4-4436-b7df-dd197be5ea46 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:30.385966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa176b87-7bf4-4436-b7df-dd197be5ea46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:30.413955 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-fa176b87-7bf4-4436-b7df-dd197be5ea46 None None] MFA Rules not processed for user `dbea161f89174362ab1bd3231b582977`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:30.415179 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-fa176b87-7bf4-4436-b7df-dd197be5ea46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:30.477691 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 979/1972] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:30.488848 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe506f3-b0d4-49db-ad96-5539899df533 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:30.490517 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-abe506f3-b0d4-49db-ad96-5539899df533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:30.545801 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe506f3-b0d4-49db-ad96-5539899df533 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:30.549365 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d86cf-b311-4045-aa95-2be5590c7991 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:30.549993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d86cf-b311-4045-aa95-2be5590c7991 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:30.550273 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d86cf-b311-4045-aa95-2be5590c7991 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:30.550542 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d86cf-b311-4045-aa95-2be5590c7991 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:30.558670 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d86cf-b311-4045-aa95-2be5590c7991 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:30.560106 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d86cf-b311-4045-aa95-2be5590c7991 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:30.568728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 994/1973] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:36:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:30.593872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa176b87-7bf4-4436-b7df-dd197be5ea46 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:30.654547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa176b87-7bf4-4436-b7df-dd197be5ea46 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:30.659252 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1371159c-17ed-49bb-afd4-95ec285d13b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:30.660213 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371159c-17ed-49bb-afd4-95ec285d13b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:30.660569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371159c-17ed-49bb-afd4-95ec285d13b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:30.660852 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371159c-17ed-49bb-afd4-95ec285d13b5 admin admin] PATH_INFO: `/v3/users/dbea161f89174362ab1bd3231b582977` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:30.662082 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1371159c-17ed-49bb-afd4-95ec285d13b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbea161f89174362ab1bd3231b582977)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:30.664124 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1371159c-17ed-49bb-afd4-95ec285d13b5 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:30.890205 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1371159c-17ed-49bb-afd4-95ec285d13b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbea161f89174362ab1bd3231b582977'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:30.981104 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1371159c-17ed-49bb-afd4-95ec285d13b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbea161f89174362ab1bd3231b582977'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:30.981727 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1371159c-17ed-49bb-afd4-95ec285d13b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbea161f89174362ab1bd3231b582977'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:30.983287 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:30.989865 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1371159c-17ed-49bb-afd4-95ec285d13b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbea161f89174362ab1bd3231b582977'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:30.990539 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1371159c-17ed-49bb-afd4-95ec285d13b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbea161f89174362ab1bd3231b582977'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:31.010017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 980/1974] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:36:30 2022] DELETE /identity/v3/users/dbea161f89174362ab1bd3231b582977 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:31.014514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fc0a6-1bce-4a2b-b591-215aecbcd6b6 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:31.014794 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fc0a6-1bce-4a2b-b591-215aecbcd6b6 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:31.015111 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fc0a6-1bce-4a2b-b591-215aecbcd6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:31.033255 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-762fc0a6-1bce-4a2b-b591-215aecbcd6b6 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:31.034480 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-762fc0a6-1bce-4a2b-b591-215aecbcd6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:31.103242 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 995/1975] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:31.114046 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1371159c-17ed-49bb-afd4-95ec285d13b5 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:31.115621 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1371159c-17ed-49bb-afd4-95ec285d13b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:31.179439 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1371159c-17ed-49bb-afd4-95ec285d13b5 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:31.183183 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e56ae-d1c3-4c90-bc60-dff9b5839289 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:31.183893 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e56ae-d1c3-4c90-bc60-dff9b5839289 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:31.184179 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e56ae-d1c3-4c90-bc60-dff9b5839289 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:31.184485 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e56ae-d1c3-4c90-bc60-dff9b5839289 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:31.195998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e56ae-d1c3-4c90-bc60-dff9b5839289 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:31.197536 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e56ae-d1c3-4c90-bc60-dff9b5839289 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:31.207827 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 981/1976] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:36:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:31.610683 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fc0a6-1bce-4a2b-b591-215aecbcd6b6 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:31.612193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-762fc0a6-1bce-4a2b-b591-215aecbcd6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:31.624578 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fc0a6-1bce-4a2b-b591-215aecbcd6b6 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:31.629223 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee333654-c683-42b9-8b8c-ebf64af57158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:31.630087 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee333654-c683-42b9-8b8c-ebf64af57158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:31.630367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee333654-c683-42b9-8b8c-ebf64af57158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:31.630640 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee333654-c683-42b9-8b8c-ebf64af57158 admin admin] PATH_INFO: `/v3/projects/5c0f8424568846938ab720101b7cf794` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:31.631922 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee333654-c683-42b9-8b8c-ebf64af57158 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c0f8424568846938ab720101b7cf794)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:31.633906 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee333654-c683-42b9-8b8c-ebf64af57158 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:31.716251 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:31.728875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ee333654-c683-42b9-8b8c-ebf64af57158 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c0f8424568846938ab720101b7cf794', 'request_id': 'req-ee333654-c683-42b9-8b8c-ebf64af57158'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:31.804240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ee333654-c683-42b9-8b8c-ebf64af57158 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c0f8424568846938ab720101b7cf794', 'request_id': 'req-ee333654-c683-42b9-8b8c-ebf64af57158'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:31.810508 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ee333654-c683-42b9-8b8c-ebf64af57158 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c0f8424568846938ab720101b7cf794', 'request_id': 'req-ee333654-c683-42b9-8b8c-ebf64af57158'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:31.821682 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ee333654-c683-42b9-8b8c-ebf64af57158 admin admin] The token cache is being invalidate because project 5c0f8424568846938ab720101b7cf794 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:36:31.821966 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ee333654-c683-42b9-8b8c-ebf64af57158 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:31.822629 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ee333654-c683-42b9-8b8c-ebf64af57158 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:31.824200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 996/1977] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:36:31 2022] DELETE /identity/v3/projects/5c0f8424568846938ab720101b7cf794 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:31.829793 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375c33e-09a1-47ec-8303-221cdb761e15 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:31.830269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375c33e-09a1-47ec-8303-221cdb761e15 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:31.830365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375c33e-09a1-47ec-8303-221cdb761e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:31.856999 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-2375c33e-09a1-47ec-8303-221cdb761e15 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:31.858517 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-2375c33e-09a1-47ec-8303-221cdb761e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:31.927436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 982/1978] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:36:31.934753 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c9a33-60e4-4e25-bb11-65a40e3ef37c None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:31.935029 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c9a33-60e4-4e25-bb11-65a40e3ef37c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:31.935309 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c9a33-60e4-4e25-bb11-65a40e3ef37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:31.958659 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-ef5c9a33-60e4-4e25-bb11-65a40e3ef37c None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:31.963838 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ef5c9a33-60e4-4e25-bb11-65a40e3ef37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:31.980979 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 997/1979] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:36:31.986055 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375c33e-09a1-47ec-8303-221cdb761e15 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:31.999890 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375c33e-09a1-47ec-8303-221cdb761e15 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:32.007243 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c18781-cde5-4e38-8b1d-52d13e9fbc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:32.009600 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c18781-cde5-4e38-8b1d-52d13e9fbc2b admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:32.009600 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c18781-cde5-4e38-8b1d-52d13e9fbc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:32.009600 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c18781-cde5-4e38-8b1d-52d13e9fbc2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:32.010796 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c18781-cde5-4e38-8b1d-52d13e9fbc2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:36:32.010796 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c18781-cde5-4e38-8b1d-52d13e9fbc2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:32.013744 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c18781-cde5-4e38-8b1d-52d13e9fbc2b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:32.033316 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 983/1980] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:36:32.037245 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c9a33-60e4-4e25-bb11-65a40e3ef37c None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:32.047690 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c9a33-60e4-4e25-bb11-65a40e3ef37c None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:32.052694 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba5d8b-328a-479b-8bfb-c899889e55d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:32.053396 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba5d8b-328a-479b-8bfb-c899889e55d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:32.053725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba5d8b-328a-479b-8bfb-c899889e55d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:32.057088 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba5d8b-328a-479b-8bfb-c899889e55d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:32.057088 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba5d8b-328a-479b-8bfb-c899889e55d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:32.057088 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ba5d8b-328a-479b-8bfb-c899889e55d5 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:32.144492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 998/1981] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:36:32 2022] POST /identity/v3/projects => generated 399 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:36:32.150707 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c18781-cde5-4e38-8b1d-52d13e9fbc2b admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:32.161893 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c18781-cde5-4e38-8b1d-52d13e9fbc2b admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:32.163448 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba5d8b-328a-479b-8bfb-c899889e55d5 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:32.165634 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-25ba5d8b-328a-479b-8bfb-c899889e55d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:32.245217 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551d638-7da3-499e-a807-7fa608681256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:32.245750 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551d638-7da3-499e-a807-7fa608681256 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:32.246036 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551d638-7da3-499e-a807-7fa608681256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:32.246315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551d638-7da3-499e-a807-7fa608681256 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:32.247177 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551d638-7da3-499e-a807-7fa608681256 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:32.249232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9551d638-7da3-499e-a807-7fa608681256 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:32.258115 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:32.277632 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ba5d8b-328a-479b-8bfb-c899889e55d5 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:32.282622 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba185c-91a8-4106-8717-f2c5e7d50465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:32.283625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba185c-91a8-4106-8717-f2c5e7d50465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:32.283982 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba185c-91a8-4106-8717-f2c5e7d50465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:32.284331 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba185c-91a8-4106-8717-f2c5e7d50465 admin admin] PATH_INFO: `/v3/users/b465ef82dcd64ebda241be20c05ba344` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:32.285695 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba185c-91a8-4106-8717-f2c5e7d50465 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b465ef82dcd64ebda241be20c05ba344)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:32.287758 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba185c-91a8-4106-8717-f2c5e7d50465 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:32.391051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 984/1982] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:36:32 2022] POST /identity/v3/users => generated 300 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:36:32.396520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551d638-7da3-499e-a807-7fa608681256 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:32.406146 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551d638-7da3-499e-a807-7fa608681256 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:32.410716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2981e7-eb48-46f3-b4bf-af774b297ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:32.411409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2981e7-eb48-46f3-b4bf-af774b297ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:32.411686 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2981e7-eb48-46f3-b4bf-af774b297ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:32.411973 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2981e7-eb48-46f3-b4bf-af774b297ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:32.412821 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2981e7-eb48-46f3-b4bf-af774b297ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:32.415025 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2981e7-eb48-46f3-b4bf-af774b297ce0 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:32.427283 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 985/1983] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:36:32.432209 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2981e7-eb48-46f3-b4bf-af774b297ce0 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:32.443402 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2981e7-eb48-46f3-b4bf-af774b297ce0 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:32.452697 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85921a2-a1f2-4a57-a0ab-2c3b1bd9bddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:32.454205 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85921a2-a1f2-4a57-a0ab-2c3b1bd9bddb admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:32.454548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85921a2-a1f2-4a57-a0ab-2c3b1bd9bddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:32.454824 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85921a2-a1f2-4a57-a0ab-2c3b1bd9bddb admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:32.456411 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85921a2-a1f2-4a57-a0ab-2c3b1bd9bddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:32.458137 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85921a2-a1f2-4a57-a0ab-2c3b1bd9bddb admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:32.469733 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 986/1984] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:32.473751 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85921a2-a1f2-4a57-a0ab-2c3b1bd9bddb admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:32.486795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85921a2-a1f2-4a57-a0ab-2c3b1bd9bddb admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:32.492586 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd27835-37a4-45c4-8ffc-0a2e64b1d8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:32.493515 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd27835-37a4-45c4-8ffc-0a2e64b1d8d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:32.493792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd27835-37a4-45c4-8ffc-0a2e64b1d8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:32.494056 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd27835-37a4-45c4-8ffc-0a2e64b1d8d9 admin admin] PATH_INFO: `/v3/projects/b3424b242afe48229284fe05a4ce04ed/users/6f4ffcc57da54eb4bafb6382965aa901/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:32.512572 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd27835-37a4-45c4-8ffc-0a2e64b1d8d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3424b242afe48229284fe05a4ce04ed, user_id=6f4ffcc57da54eb4bafb6382965aa901, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:32.516016 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd27835-37a4-45c4-8ffc-0a2e64b1d8d9 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:32.526082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a4ba185c-91a8-4106-8717-f2c5e7d50465 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b465ef82dcd64ebda241be20c05ba344'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:32.598652 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 987/1985] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:36:32 2022] PUT /identity/v3/projects/b3424b242afe48229284fe05a4ce04ed/users/6f4ffcc57da54eb4bafb6382965aa901/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:32.609019 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a604fe-2938-4877-9392-ac7fb45264f0 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:32.609748 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a604fe-2938-4877-9392-ac7fb45264f0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:32.610032 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a604fe-2938-4877-9392-ac7fb45264f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:32.635581 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a4ba185c-91a8-4106-8717-f2c5e7d50465 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b465ef82dcd64ebda241be20c05ba344'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:32.639960 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-95a604fe-2938-4877-9392-ac7fb45264f0 None None] MFA Rules not processed for user `6f4ffcc57da54eb4bafb6382965aa901`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:32.641487 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-95a604fe-2938-4877-9392-ac7fb45264f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:32.650698 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a4ba185c-91a8-4106-8717-f2c5e7d50465 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b465ef82dcd64ebda241be20c05ba344'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:32.651584 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a4ba185c-91a8-4106-8717-f2c5e7d50465 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b465ef82dcd64ebda241be20c05ba344'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:32.659250 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a4ba185c-91a8-4106-8717-f2c5e7d50465 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b465ef82dcd64ebda241be20c05ba344'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:32.668558 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 999/1986] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:36:32 2022] DELETE /identity/v3/users/b465ef82dcd64ebda241be20c05ba344 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:32.673017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22172863-178c-4a43-9f08-955050edfd2c None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:32.677068 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22172863-178c-4a43-9f08-955050edfd2c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:32.677068 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22172863-178c-4a43-9f08-955050edfd2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:32.697654 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-22172863-178c-4a43-9f08-955050edfd2c None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:32.699230 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-22172863-178c-4a43-9f08-955050edfd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:32.705353 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:32.722572 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 988/1987] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:32 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:36:32.735520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a604fe-2938-4877-9392-ac7fb45264f0 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:32.738015 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-95a604fe-2938-4877-9392-ac7fb45264f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:32.788444 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1000/1988] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:32.798396 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22172863-178c-4a43-9f08-955050edfd2c None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:32.799914 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-22172863-178c-4a43-9f08-955050edfd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:32.822412 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a604fe-2938-4877-9392-ac7fb45264f0 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:32.879038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22172863-178c-4a43-9f08-955050edfd2c None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:32.887188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128b567-8d23-4871-8bc7-cf5e212a1796 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:32.887188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128b567-8d23-4871-8bc7-cf5e212a1796 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:32.887188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128b567-8d23-4871-8bc7-cf5e212a1796 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:32.887188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128b567-8d23-4871-8bc7-cf5e212a1796 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:32.887861 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab016c58-e370-45b6-a5e4-0bbe43f46517 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:32.887861 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab016c58-e370-45b6-a5e4-0bbe43f46517 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:32.887861 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab016c58-e370-45b6-a5e4-0bbe43f46517 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:32.887861 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab016c58-e370-45b6-a5e4-0bbe43f46517 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:32.890613 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:32.901694 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab016c58-e370-45b6-a5e4-0bbe43f46517 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:32.903219 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab016c58-e370-45b6-a5e4-0bbe43f46517 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:32.913952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1001/1989] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:36:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:32.914765 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab016c58-e370-45b6-a5e4-0bbe43f46517 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:32.916071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ab016c58-e370-45b6-a5e4-0bbe43f46517 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:32.929582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab016c58-e370-45b6-a5e4-0bbe43f46517 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:32.936053 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64e80e-5c45-4201-a5d3-a818f8bb1c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:32.937434 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64e80e-5c45-4201-a5d3-a818f8bb1c98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:32.937790 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64e80e-5c45-4201-a5d3-a818f8bb1c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:32.938136 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64e80e-5c45-4201-a5d3-a818f8bb1c98 admin admin] PATH_INFO: `/v3/users/8584a3f99b0144979f69536f7cc86851` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:32.940251 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f64e80e-5c45-4201-a5d3-a818f8bb1c98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8584a3f99b0144979f69536f7cc86851)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:32.942601 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f64e80e-5c45-4201-a5d3-a818f8bb1c98 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:32.959781 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128b567-8d23-4871-8bc7-cf5e212a1796 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:32.961834 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128b567-8d23-4871-8bc7-cf5e212a1796 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:32.974776 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 989/1990] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:36:32 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:36:33.159752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-6f64e80e-5c45-4201-a5d3-a818f8bb1c98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8584a3f99b0144979f69536f7cc86851'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:33.267132 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-6f64e80e-5c45-4201-a5d3-a818f8bb1c98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8584a3f99b0144979f69536f7cc86851'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:33.287319 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-6f64e80e-5c45-4201-a5d3-a818f8bb1c98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8584a3f99b0144979f69536f7cc86851'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:33.289447 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-6f64e80e-5c45-4201-a5d3-a818f8bb1c98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8584a3f99b0144979f69536f7cc86851'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:33.302079 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-6f64e80e-5c45-4201-a5d3-a818f8bb1c98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8584a3f99b0144979f69536f7cc86851'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:33.310376 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1002/1991] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:36:32 2022] DELETE /identity/v3/users/8584a3f99b0144979f69536f7cc86851 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:33.586000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128b567-8d23-4871-8bc7-cf5e212a1796 service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:33.586220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64e80e-5c45-4201-a5d3-a818f8bb1c98 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:33.587815 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6f64e80e-5c45-4201-a5d3-a818f8bb1c98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:33.603884 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128b567-8d23-4871-8bc7-cf5e212a1796 service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:33.640396 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:33.684435 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4f542-5153-4e01-8738-3b6f2c01cef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:33.685308 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4f542-5153-4e01-8738-3b6f2c01cef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:33.685587 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4f542-5153-4e01-8738-3b6f2c01cef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:33.685860 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4f542-5153-4e01-8738-3b6f2c01cef9 admin admin] PATH_INFO: `/v3/projects/e3b236cd8e9049e6acdeb545f5d3382d` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:33.687067 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b4f542-5153-4e01-8738-3b6f2c01cef9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3b236cd8e9049e6acdeb545f5d3382d)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:33.688078 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64e80e-5c45-4201-a5d3-a818f8bb1c98 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:33.689008 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b4f542-5153-4e01-8738-3b6f2c01cef9 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:33.692757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb63d79-4102-405d-8351-30d9e6fa5a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:33.693635 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb63d79-4102-405d-8351-30d9e6fa5a65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:33.694485 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb63d79-4102-405d-8351-30d9e6fa5a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:33.694752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb63d79-4102-405d-8351-30d9e6fa5a65 admin admin] PATH_INFO: `/v3/projects/1e2d7ad7477240128353e496566522d4` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:33.696006 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb63d79-4102-405d-8351-30d9e6fa5a65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e2d7ad7477240128353e496566522d4)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:33.698224 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb63d79-4102-405d-8351-30d9e6fa5a65 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:33.837751 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a5b4f542-5153-4e01-8738-3b6f2c01cef9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3b236cd8e9049e6acdeb545f5d3382d', 'request_id': 'req-a5b4f542-5153-4e01-8738-3b6f2c01cef9'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:33.837925 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-6eb63d79-4102-405d-8351-30d9e6fa5a65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e2d7ad7477240128353e496566522d4', 'request_id': 'req-6eb63d79-4102-405d-8351-30d9e6fa5a65'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:33.942329 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a5b4f542-5153-4e01-8738-3b6f2c01cef9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3b236cd8e9049e6acdeb545f5d3382d', 'request_id': 'req-a5b4f542-5153-4e01-8738-3b6f2c01cef9'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:33.950402 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a5b4f542-5153-4e01-8738-3b6f2c01cef9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3b236cd8e9049e6acdeb545f5d3382d', 'request_id': 'req-a5b4f542-5153-4e01-8738-3b6f2c01cef9'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:33.971513 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a5b4f542-5153-4e01-8738-3b6f2c01cef9 admin admin] The token cache is being invalidate because project e3b236cd8e9049e6acdeb545f5d3382d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:36:33.971788 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a5b4f542-5153-4e01-8738-3b6f2c01cef9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:33.972456 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a5b4f542-5153-4e01-8738-3b6f2c01cef9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:33.974507 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 990/1992] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:36:33 2022] DELETE /identity/v3/projects/e3b236cd8e9049e6acdeb545f5d3382d => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:33.980325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbc969-c345-4400-be8a-88dcf2f7c44f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:33.980613 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbc969-c345-4400-be8a-88dcf2f7c44f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:33.980891 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbc969-c345-4400-be8a-88dcf2f7c44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:33.999646 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e6bbc969-c345-4400-be8a-88dcf2f7c44f None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:34.000956 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e6bbc969-c345-4400-be8a-88dcf2f7c44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:34.032441 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:34.060217 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 991/1993] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:36:34.067962 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa422e-52ca-4c47-bbae-fe14930732c6 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:34.068256 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa422e-52ca-4c47-bbae-fe14930732c6 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:34.068537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa422e-52ca-4c47-bbae-fe14930732c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:34.075661 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-6eb63d79-4102-405d-8351-30d9e6fa5a65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e2d7ad7477240128353e496566522d4', 'request_id': 'req-6eb63d79-4102-405d-8351-30d9e6fa5a65'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:34.082660 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-6eb63d79-4102-405d-8351-30d9e6fa5a65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e2d7ad7477240128353e496566522d4', 'request_id': 'req-6eb63d79-4102-405d-8351-30d9e6fa5a65'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:34.098916 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-6eb63d79-4102-405d-8351-30d9e6fa5a65 admin admin] The token cache is being invalidate because project 1e2d7ad7477240128353e496566522d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:36:34.099376 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-6eb63d79-4102-405d-8351-30d9e6fa5a65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:34.100361 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-6eb63d79-4102-405d-8351-30d9e6fa5a65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:34.102076 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1003/1994] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:36:33 2022] DELETE /identity/v3/projects/1e2d7ad7477240128353e496566522d4 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:34.104187 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-81fa422e-52ca-4c47-bbae-fe14930732c6 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:34.105906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-81fa422e-52ca-4c47-bbae-fe14930732c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:34.106029 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb63d79-4102-405d-8351-30d9e6fa5a65 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:34.112241 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6eb63d79-4102-405d-8351-30d9e6fa5a65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:34.123122 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 992/1995] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:36:34.129660 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa422e-52ca-4c47-bbae-fe14930732c6 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:34.133728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb63d79-4102-405d-8351-30d9e6fa5a65 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:34.140892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c0eb3-5e42-42f1-9f2d-0d3d5db8ccb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:34.141508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa422e-52ca-4c47-bbae-fe14930732c6 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:34.141781 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c0eb3-5e42-42f1-9f2d-0d3d5db8ccb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:34.142069 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c0eb3-5e42-42f1-9f2d-0d3d5db8ccb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:34.142349 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c0eb3-5e42-42f1-9f2d-0d3d5db8ccb6 admin admin] PATH_INFO: `/v3/users/2578e2a86ac04520847d3a888ca2e3e1` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:34.143809 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004c0eb3-5e42-42f1-9f2d-0d3d5db8ccb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2578e2a86ac04520847d3a888ca2e3e1)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:34.146481 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004c0eb3-5e42-42f1-9f2d-0d3d5db8ccb6 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:34.147486 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081cba07-9f65-4797-af87-c2bf711356e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:34.148511 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cba07-9f65-4797-af87-c2bf711356e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:34.148873 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cba07-9f65-4797-af87-c2bf711356e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:34.149506 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cba07-9f65-4797-af87-c2bf711356e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:34.150550 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081cba07-9f65-4797-af87-c2bf711356e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:36:34.151305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081cba07-9f65-4797-af87-c2bf711356e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:34.154512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081cba07-9f65-4797-af87-c2bf711356e0 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:34.178911 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 993/1996] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:36:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:36:34.180560 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081cba07-9f65-4797-af87-c2bf711356e0 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:34.191226 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081cba07-9f65-4797-af87-c2bf711356e0 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:34.196155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9474a-91db-42cf-b608-60eabdaf9cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:34.196891 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9474a-91db-42cf-b608-60eabdaf9cb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:34.197481 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9474a-91db-42cf-b608-60eabdaf9cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:34.197943 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a9474a-91db-42cf-b608-60eabdaf9cb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:34.198754 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a9474a-91db-42cf-b608-60eabdaf9cb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:34.201628 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a9474a-91db-42cf-b608-60eabdaf9cb0 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:34.319176 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 994/1997] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:36:34 2022] POST /identity/v3/projects => generated 393 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:36:34.320251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9474a-91db-42cf-b608-60eabdaf9cb0 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:34.344247 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-004c0eb3-5e42-42f1-9f2d-0d3d5db8ccb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2578e2a86ac04520847d3a888ca2e3e1'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:34.388863 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a9474a-91db-42cf-b608-60eabdaf9cb0 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:34.394039 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058ed5e-b66b-44aa-b402-13695061317e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:34.394715 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058ed5e-b66b-44aa-b402-13695061317e admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:34.395228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058ed5e-b66b-44aa-b402-13695061317e admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:34.395429 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058ed5e-b66b-44aa-b402-13695061317e admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:34.396144 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058ed5e-b66b-44aa-b402-13695061317e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:34.398048 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058ed5e-b66b-44aa-b402-13695061317e admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:34.424212 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-004c0eb3-5e42-42f1-9f2d-0d3d5db8ccb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2578e2a86ac04520847d3a888ca2e3e1'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:34.437427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-004c0eb3-5e42-42f1-9f2d-0d3d5db8ccb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2578e2a86ac04520847d3a888ca2e3e1'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:34.438068 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-004c0eb3-5e42-42f1-9f2d-0d3d5db8ccb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2578e2a86ac04520847d3a888ca2e3e1'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:34.444957 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-004c0eb3-5e42-42f1-9f2d-0d3d5db8ccb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2578e2a86ac04520847d3a888ca2e3e1'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:34.453435 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1004/1998] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:36:34 2022] DELETE /identity/v3/users/2578e2a86ac04520847d3a888ca2e3e1 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:36:34.519289 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 995/1999] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:36:34 2022] POST /identity/v3/users => generated 297 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:36:34.520757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c0eb3-5e42-42f1-9f2d-0d3d5db8ccb6 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:34.522986 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-004c0eb3-5e42-42f1-9f2d-0d3d5db8ccb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:34.556915 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:34.589612 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004c0eb3-5e42-42f1-9f2d-0d3d5db8ccb6 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:34.594638 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652f7ff-1465-421a-b7bd-3eacd001862e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:34.595627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652f7ff-1465-421a-b7bd-3eacd001862e admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:34.595918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652f7ff-1465-421a-b7bd-3eacd001862e admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:34.596192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652f7ff-1465-421a-b7bd-3eacd001862e admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:34.596977 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c652f7ff-1465-421a-b7bd-3eacd001862e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:34.598571 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c652f7ff-1465-421a-b7bd-3eacd001862e admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:34.609191 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1005/2000] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:34.613242 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058ed5e-b66b-44aa-b402-13695061317e admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:34.624288 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058ed5e-b66b-44aa-b402-13695061317e admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:34.629022 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e62d1-898f-4240-a886-8b0ce85e0167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:34.629840 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e62d1-898f-4240-a886-8b0ce85e0167 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:34.630128 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e62d1-898f-4240-a886-8b0ce85e0167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:34.630406 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e62d1-898f-4240-a886-8b0ce85e0167 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:34.631169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e62d1-898f-4240-a886-8b0ce85e0167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:34.632798 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e62d1-898f-4240-a886-8b0ce85e0167 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:34.647413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 996/2001] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:34.648728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652f7ff-1465-421a-b7bd-3eacd001862e admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:34.659057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c652f7ff-1465-421a-b7bd-3eacd001862e admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:34.671365 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84771c25-8b0f-4ffc-9f14-5fdefb6e83c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:34.672307 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84771c25-8b0f-4ffc-9f14-5fdefb6e83c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:34.672593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84771c25-8b0f-4ffc-9f14-5fdefb6e83c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:34.672865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84771c25-8b0f-4ffc-9f14-5fdefb6e83c9 admin admin] PATH_INFO: `/v3/projects/03b1335715234895a0c1a555b22ee8d3/users/b2671aaae84644acb89c3c3595bb5673/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:34.689268 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84771c25-8b0f-4ffc-9f14-5fdefb6e83c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03b1335715234895a0c1a555b22ee8d3, user_id=b2671aaae84644acb89c3c3595bb5673, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:34.691958 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84771c25-8b0f-4ffc-9f14-5fdefb6e83c9 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:34.796458 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1006/2002] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:36:34 2022] PUT /identity/v3/projects/03b1335715234895a0c1a555b22ee8d3/users/b2671aaae84644acb89c3c3595bb5673/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:34.801573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452543ff-cedc-4884-846e-7b36c343fb39 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:34.801868 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452543ff-cedc-4884-846e-7b36c343fb39 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:34.802173 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452543ff-cedc-4884-846e-7b36c343fb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:34.819789 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-452543ff-cedc-4884-846e-7b36c343fb39 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:34.820997 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-452543ff-cedc-4884-846e-7b36c343fb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:34.879372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 997/2003] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:34.890408 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84771c25-8b0f-4ffc-9f14-5fdefb6e83c9 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:34.892507 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-84771c25-8b0f-4ffc-9f14-5fdefb6e83c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:34.920116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452543ff-cedc-4884-846e-7b36c343fb39 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:34.921735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-452543ff-cedc-4884-846e-7b36c343fb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:34.935105 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452543ff-cedc-4884-846e-7b36c343fb39 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:34.940418 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b511e9db-0ba1-41b9-8c54-57073d80596b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:34.941291 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511e9db-0ba1-41b9-8c54-57073d80596b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:34.941528 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511e9db-0ba1-41b9-8c54-57073d80596b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:34.941794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b511e9db-0ba1-41b9-8c54-57073d80596b admin admin] PATH_INFO: `/v3/projects/43f139ff0fba435f84e6b5d203a57c54` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:34.942973 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b511e9db-0ba1-41b9-8c54-57073d80596b admin admin] RBAC: Authorizing `identity:delete_project(project_id=43f139ff0fba435f84e6b5d203a57c54)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:34.950249 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b511e9db-0ba1-41b9-8c54-57073d80596b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:34.955402 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:34.969203 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84771c25-8b0f-4ffc-9f14-5fdefb6e83c9 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:34.973043 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1dd8a3-a860-4660-8718-87f91213a7b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:34.973769 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1dd8a3-a860-4660-8718-87f91213a7b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:34.974024 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1dd8a3-a860-4660-8718-87f91213a7b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:34.974301 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1dd8a3-a860-4660-8718-87f91213a7b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:34.983577 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1dd8a3-a860-4660-8718-87f91213a7b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:34.985049 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1dd8a3-a860-4660-8718-87f91213a7b3 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:34.996492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1007/2004] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:36:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:35.056351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b511e9db-0ba1-41b9-8c54-57073d80596b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43f139ff0fba435f84e6b5d203a57c54', 'request_id': 'req-b511e9db-0ba1-41b9-8c54-57073d80596b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:35.116253 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b511e9db-0ba1-41b9-8c54-57073d80596b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43f139ff0fba435f84e6b5d203a57c54', 'request_id': 'req-b511e9db-0ba1-41b9-8c54-57073d80596b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:35.124191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b511e9db-0ba1-41b9-8c54-57073d80596b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43f139ff0fba435f84e6b5d203a57c54', 'request_id': 'req-b511e9db-0ba1-41b9-8c54-57073d80596b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:35.141216 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b511e9db-0ba1-41b9-8c54-57073d80596b admin admin] The token cache is being invalidate because project 43f139ff0fba435f84e6b5d203a57c54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:36:35.141601 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b511e9db-0ba1-41b9-8c54-57073d80596b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:35.142398 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b511e9db-0ba1-41b9-8c54-57073d80596b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:35.145145 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 998/2005] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:36:34 2022] DELETE /identity/v3/projects/43f139ff0fba435f84e6b5d203a57c54 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:35.151410 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa76dbe-8194-4ee7-ac95-1fa3b67e4947 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:35.151703 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa76dbe-8194-4ee7-ac95-1fa3b67e4947 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:35.155374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa76dbe-8194-4ee7-ac95-1fa3b67e4947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:35.172894 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-2fa76dbe-8194-4ee7-ac95-1fa3b67e4947 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:35.173439 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-2fa76dbe-8194-4ee7-ac95-1fa3b67e4947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:35.257037 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1008/2006] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:35.265101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8497c-5493-4b89-8e00-9699cede14e8 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:35.265393 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8497c-5493-4b89-8e00-9699cede14e8 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:35.274280 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8497c-5493-4b89-8e00-9699cede14e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:35.284263 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-24b8497c-5493-4b89-8e00-9699cede14e8 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:35.285494 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-24b8497c-5493-4b89-8e00-9699cede14e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:35.293714 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 999/2007] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:35.298987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa76dbe-8194-4ee7-ac95-1fa3b67e4947 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:35.312203 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa76dbe-8194-4ee7-ac95-1fa3b67e4947 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:35.317094 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c61c9f-fc55-4ea5-ae8d-52e66e9d81f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:35.317832 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c61c9f-fc55-4ea5-ae8d-52e66e9d81f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:35.318208 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c61c9f-fc55-4ea5-ae8d-52e66e9d81f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:35.318550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c61c9f-fc55-4ea5-ae8d-52e66e9d81f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:35.319934 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c61c9f-fc55-4ea5-ae8d-52e66e9d81f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:36:35.320307 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c61c9f-fc55-4ea5-ae8d-52e66e9d81f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:35.321907 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c61c9f-fc55-4ea5-ae8d-52e66e9d81f2 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:35.335051 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1009/2008] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:36:35.343349 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8497c-5493-4b89-8e00-9699cede14e8 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:35.357879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8497c-5493-4b89-8e00-9699cede14e8 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:35.373174 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15735d58-39b7-4727-ae63-3839aebaf59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:35.373862 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15735d58-39b7-4727-ae63-3839aebaf59b admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:35.374145 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15735d58-39b7-4727-ae63-3839aebaf59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:35.374424 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15735d58-39b7-4727-ae63-3839aebaf59b admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:35.375165 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15735d58-39b7-4727-ae63-3839aebaf59b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:35.377390 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15735d58-39b7-4727-ae63-3839aebaf59b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:35.463812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1000/2009] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:36:35 2022] POST /identity/v3/projects => generated 381 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:36:35.468172 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c61c9f-fc55-4ea5-ae8d-52e66e9d81f2 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:35.480647 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c61c9f-fc55-4ea5-ae8d-52e66e9d81f2 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:35.511921 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:35.552943 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83763e70-92dd-4ab9-8045-41c904cc2cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:35.555197 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83763e70-92dd-4ab9-8045-41c904cc2cb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:35.555546 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83763e70-92dd-4ab9-8045-41c904cc2cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:35.555961 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83763e70-92dd-4ab9-8045-41c904cc2cb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:35.556743 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83763e70-92dd-4ab9-8045-41c904cc2cb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:35.559047 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83763e70-92dd-4ab9-8045-41c904cc2cb0 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:35.756770 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1010/2010] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:36:35 2022] POST /identity/v3/users => generated 291 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:36:35.760954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15735d58-39b7-4727-ae63-3839aebaf59b admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:35.771263 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15735d58-39b7-4727-ae63-3839aebaf59b admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:35.775784 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c47892-2deb-44b8-9391-d7bccde58db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:35.776434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c47892-2deb-44b8-9391-d7bccde58db1 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:35.776712 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c47892-2deb-44b8-9391-d7bccde58db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:35.776986 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c47892-2deb-44b8-9391-d7bccde58db1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:35.777704 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c47892-2deb-44b8-9391-d7bccde58db1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:35.779327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c47892-2deb-44b8-9391-d7bccde58db1 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:35.793280 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1001/2011] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:35.798448 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83763e70-92dd-4ab9-8045-41c904cc2cb0 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:35.809159 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83763e70-92dd-4ab9-8045-41c904cc2cb0 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:35.813906 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae48657-0e28-4a0a-a265-a0d8c0c8d624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:35.814572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae48657-0e28-4a0a-a265-a0d8c0c8d624 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:35.814886 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae48657-0e28-4a0a-a265-a0d8c0c8d624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:35.815168 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae48657-0e28-4a0a-a265-a0d8c0c8d624 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:35.815902 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae48657-0e28-4a0a-a265-a0d8c0c8d624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:35.817684 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae48657-0e28-4a0a-a265-a0d8c0c8d624 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:35.829650 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1011/2012] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:35.836229 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c47892-2deb-44b8-9391-d7bccde58db1 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:35.846151 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c47892-2deb-44b8-9391-d7bccde58db1 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:35.851342 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab096a55-44f3-4758-959d-515a6b5b0db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:35.852229 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab096a55-44f3-4758-959d-515a6b5b0db0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:35.852505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab096a55-44f3-4758-959d-515a6b5b0db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:35.852777 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab096a55-44f3-4758-959d-515a6b5b0db0 admin admin] PATH_INFO: `/v3/projects/9593b6b2f3b444f2b79dafce83bb484b/users/06c8ff616dc04459be2e15ef74459af4/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:35.868206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab096a55-44f3-4758-959d-515a6b5b0db0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9593b6b2f3b444f2b79dafce83bb484b, user_id=06c8ff616dc04459be2e15ef74459af4, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:35.870914 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab096a55-44f3-4758-959d-515a6b5b0db0 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:35.956542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1002/2013] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:36:35 2022] PUT /identity/v3/projects/9593b6b2f3b444f2b79dafce83bb484b/users/06c8ff616dc04459be2e15ef74459af4/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:35.967386 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953a0eb-9109-4dcc-9dd0-4c6a40c3bf78 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:35.967674 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953a0eb-9109-4dcc-9dd0-4c6a40c3bf78 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:35.967949 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9953a0eb-9109-4dcc-9dd0-4c6a40c3bf78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:35.996254 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-9953a0eb-9109-4dcc-9dd0-4c6a40c3bf78 None None] MFA Rules not processed for user `06c8ff616dc04459be2e15ef74459af4`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:35.997492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-9953a0eb-9109-4dcc-9dd0-4c6a40c3bf78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:36.080344 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1012/2014] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:36.088795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab096a55-44f3-4758-959d-515a6b5b0db0 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:36.160056 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab096a55-44f3-4758-959d-515a6b5b0db0 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:36.165367 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff5bff-568e-4d22-b38d-a19e876f89a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:36.166032 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff5bff-568e-4d22-b38d-a19e876f89a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:36.166310 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff5bff-568e-4d22-b38d-a19e876f89a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:36.166584 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff5bff-568e-4d22-b38d-a19e876f89a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:36.167313 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff5bff-568e-4d22-b38d-a19e876f89a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:36.169235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff5bff-568e-4d22-b38d-a19e876f89a3 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:36.337352 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1003/2015] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:36:36 2022] POST /identity/v3/projects => generated 381 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:36:36.339299 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953a0eb-9109-4dcc-9dd0-4c6a40c3bf78 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:36.352871 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9953a0eb-9109-4dcc-9dd0-4c6a40c3bf78 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:36.425754 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32126448-7cd1-4a26-924b-127614454c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:36.426544 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32126448-7cd1-4a26-924b-127614454c48 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:36.426922 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32126448-7cd1-4a26-924b-127614454c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:36.427267 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32126448-7cd1-4a26-924b-127614454c48 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:36.428045 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32126448-7cd1-4a26-924b-127614454c48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:36.430670 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32126448-7cd1-4a26-924b-127614454c48 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:36.510289 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1013/2016] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:36:36 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:36:36.514538 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff5bff-568e-4d22-b38d-a19e876f89a3 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:36.526907 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff5bff-568e-4d22-b38d-a19e876f89a3 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:36.532449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe96e8-624e-4367-bd6f-55a3514dd0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:36.532642 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe96e8-624e-4367-bd6f-55a3514dd0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:36.532925 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe96e8-624e-4367-bd6f-55a3514dd0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:36.533200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe96e8-624e-4367-bd6f-55a3514dd0e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:36.533930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe96e8-624e-4367-bd6f-55a3514dd0e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:36.535580 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe96e8-624e-4367-bd6f-55a3514dd0e6 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:36.547554 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1004/2017] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:36.551818 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32126448-7cd1-4a26-924b-127614454c48 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:36.565417 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32126448-7cd1-4a26-924b-127614454c48 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:36.570742 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf2b31-8eee-4114-8f91-7081ddc1693e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:36.571686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf2b31-8eee-4114-8f91-7081ddc1693e admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:36.574821 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf2b31-8eee-4114-8f91-7081ddc1693e admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:36.574821 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf2b31-8eee-4114-8f91-7081ddc1693e admin admin] PATH_INFO: `/v3/projects/c775c084721e4a6187b84080a70ebe8d/users/2389e99809bf4d8fa534526c2ce49722/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:36.591867 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf2b31-8eee-4114-8f91-7081ddc1693e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c775c084721e4a6187b84080a70ebe8d, user_id=2389e99809bf4d8fa534526c2ce49722, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:36.594533 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf2b31-8eee-4114-8f91-7081ddc1693e admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:36.680072 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1014/2018] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:36:36 2022] PUT /identity/v3/projects/c775c084721e4a6187b84080a70ebe8d/users/2389e99809bf4d8fa534526c2ce49722/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:36.684169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe96e8-624e-4367-bd6f-55a3514dd0e6 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:36.695274 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe96e8-624e-4367-bd6f-55a3514dd0e6 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:36.699845 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:36.778756 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c7c5c-f4ad-425a-9f26-f267ed327301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:36.779487 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c7c5c-f4ad-425a-9f26-f267ed327301 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:36.780283 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c7c5c-f4ad-425a-9f26-f267ed327301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:36.780667 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c7c5c-f4ad-425a-9f26-f267ed327301 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:36.781675 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c7c5c-f4ad-425a-9f26-f267ed327301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:36.783737 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c7c5c-f4ad-425a-9f26-f267ed327301 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:36.799373 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1005/2019] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:36.832961 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf2b31-8eee-4114-8f91-7081ddc1693e admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:36.841130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:36.845020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf2b31-8eee-4114-8f91-7081ddc1693e admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:36.854590 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb2debf-427a-4803-9d1f-91c9e2a616df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:36.855752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2debf-427a-4803-9d1f-91c9e2a616df admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:36.856138 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2debf-427a-4803-9d1f-91c9e2a616df admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:36.856512 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2debf-427a-4803-9d1f-91c9e2a616df admin admin] PATH_INFO: `/v3/projects/c775c084721e4a6187b84080a70ebe8d/users/2389e99809bf4d8fa534526c2ce49722/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:36.858656 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb2debf-427a-4803-9d1f-91c9e2a616df admin admin] RBAC: Authorizing `identity:create_grant(project_id=c775c084721e4a6187b84080a70ebe8d, user_id=2389e99809bf4d8fa534526c2ce49722, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:36.861494 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb2debf-427a-4803-9d1f-91c9e2a616df admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:36.874891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1015/2020] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:36:36 2022] PUT /identity/v3/projects/c775c084721e4a6187b84080a70ebe8d/users/2389e99809bf4d8fa534526c2ce49722/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:36.886032 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456c7ca-8417-4559-be54-296da6a9b787 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:36.886426 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456c7ca-8417-4559-be54-296da6a9b787 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:36.886902 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456c7ca-8417-4559-be54-296da6a9b787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:36.915464 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d456c7ca-8417-4559-be54-296da6a9b787 None None] MFA Rules not processed for user `2389e99809bf4d8fa534526c2ce49722`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:36.916824 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d456c7ca-8417-4559-be54-296da6a9b787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:36.953797 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb2debf-427a-4803-9d1f-91c9e2a616df admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:36.955716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-7fb2debf-427a-4803-9d1f-91c9e2a616df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:36.993838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1006/2021] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:37.006388 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456c7ca-8417-4559-be54-296da6a9b787 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:37.011333 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d456c7ca-8417-4559-be54-296da6a9b787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:37.031569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb2debf-427a-4803-9d1f-91c9e2a616df admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:37.036653 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931b655-ea5b-438b-b34d-c79525a2abf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:37.037618 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931b655-ea5b-438b-b34d-c79525a2abf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:37.038060 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931b655-ea5b-438b-b34d-c79525a2abf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:37.038175 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931b655-ea5b-438b-b34d-c79525a2abf2 admin admin] PATH_INFO: `/v3/users/5153a1b4ca604972812ea749922e9a0b` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:37.039788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931b655-ea5b-438b-b34d-c79525a2abf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5153a1b4ca604972812ea749922e9a0b)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:37.041992 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931b655-ea5b-438b-b34d-c79525a2abf2 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:37.085226 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456c7ca-8417-4559-be54-296da6a9b787 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:37.090481 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3d656-aef2-4014-9fe8-551628d5b54a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:37.091189 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3d656-aef2-4014-9fe8-551628d5b54a service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:37.091464 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3d656-aef2-4014-9fe8-551628d5b54a service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:37.091735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3d656-aef2-4014-9fe8-551628d5b54a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:37.101778 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3d656-aef2-4014-9fe8-551628d5b54a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:37.103034 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3d656-aef2-4014-9fe8-551628d5b54a service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:37.129855 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1007/2022] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:36:37 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:36:37.342258 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-c931b655-ea5b-438b-b34d-c79525a2abf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5153a1b4ca604972812ea749922e9a0b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:37.436206 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-c931b655-ea5b-438b-b34d-c79525a2abf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5153a1b4ca604972812ea749922e9a0b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:37.461960 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-c931b655-ea5b-438b-b34d-c79525a2abf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5153a1b4ca604972812ea749922e9a0b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:37.462790 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-c931b655-ea5b-438b-b34d-c79525a2abf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5153a1b4ca604972812ea749922e9a0b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:37.472140 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-c931b655-ea5b-438b-b34d-c79525a2abf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5153a1b4ca604972812ea749922e9a0b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:37.479955 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1016/2023] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:36:36 2022] DELETE /identity/v3/users/5153a1b4ca604972812ea749922e9a0b => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:36:37.484629 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa083ae9-b2b3-4fb8-8ed1-f6fdd6a0de4d None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:37.485009 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa083ae9-b2b3-4fb8-8ed1-f6fdd6a0de4d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:37.485358 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa083ae9-b2b3-4fb8-8ed1-f6fdd6a0de4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:37.520546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-fa083ae9-b2b3-4fb8-8ed1-f6fdd6a0de4d None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:37.521810 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-fa083ae9-b2b3-4fb8-8ed1-f6fdd6a0de4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:37.567410 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:37.613942 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1008/2024] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:37.628506 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931b655-ea5b-438b-b34d-c79525a2abf2 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:37.629962 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c931b655-ea5b-438b-b34d-c79525a2abf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:37.721971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931b655-ea5b-438b-b34d-c79525a2abf2 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:37.721971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da784973-f1bd-4394-beee-3f1b7e007b73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:37.721971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da784973-f1bd-4394-beee-3f1b7e007b73 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:37.721971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da784973-f1bd-4394-beee-3f1b7e007b73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:37.722284 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da784973-f1bd-4394-beee-3f1b7e007b73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:37.729974 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da784973-f1bd-4394-beee-3f1b7e007b73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:37.731583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da784973-f1bd-4394-beee-3f1b7e007b73 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:37.742818 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1017/2025] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:36:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:38.747441 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa083ae9-b2b3-4fb8-8ed1-f6fdd6a0de4d None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:38.749028 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-fa083ae9-b2b3-4fb8-8ed1-f6fdd6a0de4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:38.766790 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa083ae9-b2b3-4fb8-8ed1-f6fdd6a0de4d None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:38.776973 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b5148-d554-4584-8a51-1a8fcf0dd3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:38.780558 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b5148-d554-4584-8a51-1a8fcf0dd3d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:38.787044 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b5148-d554-4584-8a51-1a8fcf0dd3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:38.787044 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b5148-d554-4584-8a51-1a8fcf0dd3d4 admin admin] PATH_INFO: `/v3/projects/40ff1c22c2bf4d8c8e180182751d0b36` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:38.787044 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b5148-d554-4584-8a51-1a8fcf0dd3d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40ff1c22c2bf4d8c8e180182751d0b36)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:38.787044 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b5148-d554-4584-8a51-1a8fcf0dd3d4 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:38.789956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da784973-f1bd-4394-beee-3f1b7e007b73 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:38.792336 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-da784973-f1bd-4394-beee-3f1b7e007b73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:38.812159 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:38.892870 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da784973-f1bd-4394-beee-3f1b7e007b73 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:38.898556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f33a86-da26-4c58-9950-5cbd371a7943 tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2389e99809bf4d8fa534526c2ce49722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c775c084721e4a6187b84080a70ebe8d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:38.899555 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f33a86-da26-4c58-9950-5cbd371a7943 tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:38.899856 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f33a86-da26-4c58-9950-5cbd371a7943 tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:38.900138 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f33a86-da26-4c58-9950-5cbd371a7943 tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:38.901250 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f33a86-da26-4c58-9950-5cbd371a7943 tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:36:38.901713 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f33a86-da26-4c58-9950-5cbd371a7943 tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:38.904450 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f33a86-da26-4c58-9950-5cbd371a7943 tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:38.927872 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1018/2026] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:36:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:36:38.932585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f33a86-da26-4c58-9950-5cbd371a7943 tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:38.944850 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f33a86-da26-4c58-9950-5cbd371a7943 tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:38.952454 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af2049-e654-4ac2-a6e8-de9bc6f46597 tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2389e99809bf4d8fa534526c2ce49722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c775c084721e4a6187b84080a70ebe8d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:38.952957 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af2049-e654-4ac2-a6e8-de9bc6f46597 tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:38.953254 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af2049-e654-4ac2-a6e8-de9bc6f46597 tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:38.953533 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af2049-e654-4ac2-a6e8-de9bc6f46597 tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:38.954251 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af2049-e654-4ac2-a6e8-de9bc6f46597 tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:38.956924 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af2049-e654-4ac2-a6e8-de9bc6f46597 tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:38.979796 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-997b5148-d554-4584-8a51-1a8fcf0dd3d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40ff1c22c2bf4d8c8e180182751d0b36', 'request_id': 'req-997b5148-d554-4584-8a51-1a8fcf0dd3d4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:39.076615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-997b5148-d554-4584-8a51-1a8fcf0dd3d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40ff1c22c2bf4d8c8e180182751d0b36', 'request_id': 'req-997b5148-d554-4584-8a51-1a8fcf0dd3d4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:39.089138 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-997b5148-d554-4584-8a51-1a8fcf0dd3d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40ff1c22c2bf4d8c8e180182751d0b36', 'request_id': 'req-997b5148-d554-4584-8a51-1a8fcf0dd3d4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:39.090884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1019/2027] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:36:38 2022] POST /identity/v3/projects => generated 355 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:36:39.095620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af2049-e654-4ac2-a6e8-de9bc6f46597 tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:39.113791 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-997b5148-d554-4584-8a51-1a8fcf0dd3d4 admin admin] The token cache is being invalidate because project 40ff1c22c2bf4d8c8e180182751d0b36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:36:39.114330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-997b5148-d554-4584-8a51-1a8fcf0dd3d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:39.115610 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-997b5148-d554-4584-8a51-1a8fcf0dd3d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:39.117714 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1009/2028] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:36:38 2022] DELETE /identity/v3/projects/40ff1c22c2bf4d8c8e180182751d0b36 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:39.121645 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b5148-d554-4584-8a51-1a8fcf0dd3d4 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:39.123077 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-997b5148-d554-4584-8a51-1a8fcf0dd3d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:39.173638 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af2049-e654-4ac2-a6e8-de9bc6f46597 tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:39.178179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed060f6-05e8-42b9-8a9d-233ad5f17138 tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2389e99809bf4d8fa534526c2ce49722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c775c084721e4a6187b84080a70ebe8d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:39.178860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed060f6-05e8-42b9-8a9d-233ad5f17138 tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:39.179167 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed060f6-05e8-42b9-8a9d-233ad5f17138 tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:39.179531 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed060f6-05e8-42b9-8a9d-233ad5f17138 tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:39.180161 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed060f6-05e8-42b9-8a9d-233ad5f17138 tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:36:39.180468 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed060f6-05e8-42b9-8a9d-233ad5f17138 tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:39.182068 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed060f6-05e8-42b9-8a9d-233ad5f17138 tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:39.197100 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1020/2029] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:36:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:36:39.197307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b5148-d554-4584-8a51-1a8fcf0dd3d4 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:39.237201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed060f6-05e8-42b9-8a9d-233ad5f17138 tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:39.240307 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-0ed060f6-05e8-42b9-8a9d-233ad5f17138 tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:39.256132 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed060f6-05e8-42b9-8a9d-233ad5f17138 tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:39.262694 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb55289-d194-4fe0-bc1b-f9b19900994c tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2389e99809bf4d8fa534526c2ce49722', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c775c084721e4a6187b84080a70ebe8d', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:39.264058 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb55289-d194-4fe0-bc1b-f9b19900994c tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:39.264058 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb55289-d194-4fe0-bc1b-f9b19900994c tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:39.264282 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb55289-d194-4fe0-bc1b-f9b19900994c tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] PATH_INFO: `/v3/projects/d0d0020d46454693bc06a04ccc4af2ea` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:39.265359 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb55289-d194-4fe0-bc1b-f9b19900994c tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d0d0020d46454693bc06a04ccc4af2ea)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:39.267456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb55289-d194-4fe0-bc1b-f9b19900994c tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:39.274517 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd9bff-a268-4bb6-831c-78481150bee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:39.275431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd9bff-a268-4bb6-831c-78481150bee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:39.276101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd9bff-a268-4bb6-831c-78481150bee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:39.276682 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd9bff-a268-4bb6-831c-78481150bee0 admin admin] PATH_INFO: `/v3/users/590117a74b744ede8574fc98a10aacce` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:39.278158 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd9bff-a268-4bb6-831c-78481150bee0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=590117a74b744ede8574fc98a10aacce)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:39.280174 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd9bff-a268-4bb6-831c-78481150bee0 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:39.282769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:39.502902 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1bb55289-d194-4fe0-bc1b-f9b19900994c tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0d0020d46454693bc06a04ccc4af2ea', 'request_id': 'req-1bb55289-d194-4fe0-bc1b-f9b19900994c'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:39.519063 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e5cd9bff-a268-4bb6-831c-78481150bee0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '590117a74b744ede8574fc98a10aacce'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:39.613190 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1bb55289-d194-4fe0-bc1b-f9b19900994c tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0d0020d46454693bc06a04ccc4af2ea', 'request_id': 'req-1bb55289-d194-4fe0-bc1b-f9b19900994c'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:39.619534 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1bb55289-d194-4fe0-bc1b-f9b19900994c tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0d0020d46454693bc06a04ccc4af2ea', 'request_id': 'req-1bb55289-d194-4fe0-bc1b-f9b19900994c'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:39.641319 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1bb55289-d194-4fe0-bc1b-f9b19900994c tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] The token cache is being invalidate because project d0d0020d46454693bc06a04ccc4af2ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:36:39.641641 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1bb55289-d194-4fe0-bc1b-f9b19900994c tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:39.642679 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1bb55289-d194-4fe0-bc1b-f9b19900994c tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:39.658505 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1021/2030] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:36:39 2022] DELETE /identity/v3/projects/d0d0020d46454693bc06a04ccc4af2ea => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:39.679983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e5cd9bff-a268-4bb6-831c-78481150bee0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '590117a74b744ede8574fc98a10aacce'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:39.680576 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e5cd9bff-a268-4bb6-831c-78481150bee0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '590117a74b744ede8574fc98a10aacce'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:39.695553 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e5cd9bff-a268-4bb6-831c-78481150bee0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '590117a74b744ede8574fc98a10aacce'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:39.696489 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e5cd9bff-a268-4bb6-831c-78481150bee0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '590117a74b744ede8574fc98a10aacce'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:39.734505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1010/2031] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:36:39 2022] DELETE /identity/v3/users/590117a74b744ede8574fc98a10aacce => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:36:40.318607 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb55289-d194-4fe0-bc1b-f9b19900994c tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:40.323092 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1bb55289-d194-4fe0-bc1b-f9b19900994c tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:40.329641 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:40.402934 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb55289-d194-4fe0-bc1b-f9b19900994c tempest-VolumeQuotaClassesTest-1126990998 tempest-VolumeQuotaClassesTest-1126990998-project-admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:40.504926 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd9bff-a268-4bb6-831c-78481150bee0 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:40.507166 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e5cd9bff-a268-4bb6-831c-78481150bee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:40.573458 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc46f71-ca04-40b0-901d-25e9072c0201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:40.573458 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc46f71-ca04-40b0-901d-25e9072c0201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:40.573458 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc46f71-ca04-40b0-901d-25e9072c0201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:40.573458 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc46f71-ca04-40b0-901d-25e9072c0201 admin admin] PATH_INFO: `/v3/projects/6d476e8cf4584c879fd0b8ad2e05df47` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:40.575038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc46f71-ca04-40b0-901d-25e9072c0201 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d476e8cf4584c879fd0b8ad2e05df47)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:40.575038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc46f71-ca04-40b0-901d-25e9072c0201 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:40.620722 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd9bff-a268-4bb6-831c-78481150bee0 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:40.631543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f74cb-b34b-48b9-9912-2f86b91cc348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:40.632565 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f74cb-b34b-48b9-9912-2f86b91cc348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:40.632918 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f74cb-b34b-48b9-9912-2f86b91cc348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:40.633204 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f74cb-b34b-48b9-9912-2f86b91cc348 admin admin] PATH_INFO: `/v3/users/06c8ff616dc04459be2e15ef74459af4` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:40.634437 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f74cb-b34b-48b9-9912-2f86b91cc348 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06c8ff616dc04459be2e15ef74459af4)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:40.636642 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f74cb-b34b-48b9-9912-2f86b91cc348 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:40.751921 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-abc46f71-ca04-40b0-901d-25e9072c0201 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d476e8cf4584c879fd0b8ad2e05df47', 'request_id': 'req-abc46f71-ca04-40b0-901d-25e9072c0201'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:40.855886 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-abc46f71-ca04-40b0-901d-25e9072c0201 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d476e8cf4584c879fd0b8ad2e05df47', 'request_id': 'req-abc46f71-ca04-40b0-901d-25e9072c0201'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:40.876072 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-abc46f71-ca04-40b0-901d-25e9072c0201 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d476e8cf4584c879fd0b8ad2e05df47', 'request_id': 'req-abc46f71-ca04-40b0-901d-25e9072c0201'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:40.895694 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-921f74cb-b34b-48b9-9912-2f86b91cc348 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06c8ff616dc04459be2e15ef74459af4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:40.907540 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-abc46f71-ca04-40b0-901d-25e9072c0201 admin admin] The token cache is being invalidate because project 6d476e8cf4584c879fd0b8ad2e05df47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:36:40.907540 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-abc46f71-ca04-40b0-901d-25e9072c0201 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:40.907540 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-abc46f71-ca04-40b0-901d-25e9072c0201 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:40.910894 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1022/2032] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:36:40 2022] DELETE /identity/v3/projects/6d476e8cf4584c879fd0b8ad2e05df47 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:40.916936 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e33ae3-2724-4be2-8596-93c059ac8214 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:40.916936 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e33ae3-2724-4be2-8596-93c059ac8214 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:40.916936 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e33ae3-2724-4be2-8596-93c059ac8214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:40.936159 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-d3e33ae3-2724-4be2-8596-93c059ac8214 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:40.937478 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d3e33ae3-2724-4be2-8596-93c059ac8214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:41.030111 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1023/2033] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:41.032350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e81c0-0040-4e3b-b3fb-097fc8c52f76 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:41.033072 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e81c0-0040-4e3b-b3fb-097fc8c52f76 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:41.033072 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e81c0-0040-4e3b-b3fb-097fc8c52f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:41.047311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-921f74cb-b34b-48b9-9912-2f86b91cc348 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06c8ff616dc04459be2e15ef74459af4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:41.048594 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-921f74cb-b34b-48b9-9912-2f86b91cc348 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06c8ff616dc04459be2e15ef74459af4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:41.058288 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-183e81c0-0040-4e3b-b3fb-097fc8c52f76 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:41.059690 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-183e81c0-0040-4e3b-b3fb-097fc8c52f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:41.060000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-921f74cb-b34b-48b9-9912-2f86b91cc348 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06c8ff616dc04459be2e15ef74459af4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:41.061045 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-921f74cb-b34b-48b9-9912-2f86b91cc348 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06c8ff616dc04459be2e15ef74459af4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:41.072441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1024/2034] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:36:41.076688 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e81c0-0040-4e3b-b3fb-097fc8c52f76 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:41.101635 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e81c0-0040-4e3b-b3fb-097fc8c52f76 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:41.110421 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65368fb2-a951-4fee-a43e-0ea09a055928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:41.111240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65368fb2-a951-4fee-a43e-0ea09a055928 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:41.111622 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65368fb2-a951-4fee-a43e-0ea09a055928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:41.111966 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65368fb2-a951-4fee-a43e-0ea09a055928 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:41.112900 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65368fb2-a951-4fee-a43e-0ea09a055928 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:36:41.113285 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65368fb2-a951-4fee-a43e-0ea09a055928 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:41.114290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:41.116004 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65368fb2-a951-4fee-a43e-0ea09a055928 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:41.120006 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1011/2035] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:36:40 2022] DELETE /identity/v3/users/06c8ff616dc04459be2e15ef74459af4 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:41.124474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff9f24-e8f8-42be-bd72-7e275177dcf0 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:41.124765 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff9f24-e8f8-42be-bd72-7e275177dcf0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:41.125049 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff9f24-e8f8-42be-bd72-7e275177dcf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:41.137139 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1025/2036] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:36:41.141874 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65368fb2-a951-4fee-a43e-0ea09a055928 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:41.155008 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65368fb2-a951-4fee-a43e-0ea09a055928 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:41.157901 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-eaff9f24-e8f8-42be-bd72-7e275177dcf0 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:41.157901 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-eaff9f24-e8f8-42be-bd72-7e275177dcf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:41.228366 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c2ca7-2b87-4a9f-8814-d457a66790a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:41.229161 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c2ca7-2b87-4a9f-8814-d457a66790a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:41.229449 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c2ca7-2b87-4a9f-8814-d457a66790a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:41.229727 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c2ca7-2b87-4a9f-8814-d457a66790a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:41.230438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c2ca7-2b87-4a9f-8814-d457a66790a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:41.232016 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1012/2037] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:41.232423 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c2ca7-2b87-4a9f-8814-d457a66790a5 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:41.241730 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff9f24-e8f8-42be-bd72-7e275177dcf0 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:41.243684 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-eaff9f24-e8f8-42be-bd72-7e275177dcf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:41.323478 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff9f24-e8f8-42be-bd72-7e275177dcf0 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:41.325423 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58086a-0029-4758-a248-801f31eddfa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:41.326098 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58086a-0029-4758-a248-801f31eddfa1 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:41.326648 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58086a-0029-4758-a248-801f31eddfa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:41.326973 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58086a-0029-4758-a248-801f31eddfa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:41.346644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58086a-0029-4758-a248-801f31eddfa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:41.346644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58086a-0029-4758-a248-801f31eddfa1 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:41.356805 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1013/2038] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:36:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:36:41.412585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1026/2039] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:36:41 2022] POST /identity/v3/projects => generated 371 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:36:41.418093 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58086a-0029-4758-a248-801f31eddfa1 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:41.445790 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58086a-0029-4758-a248-801f31eddfa1 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:41.583325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ba9bf-df7a-44cc-935c-cd9c274e642d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:41.584784 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ba9bf-df7a-44cc-935c-cd9c274e642d admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:41.585268 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ba9bf-df7a-44cc-935c-cd9c274e642d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:41.585701 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ba9bf-df7a-44cc-935c-cd9c274e642d admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:41.586576 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86ba9bf-df7a-44cc-935c-cd9c274e642d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:41.589295 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86ba9bf-df7a-44cc-935c-cd9c274e642d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:41.696739 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1014/2040] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:36:41 2022] POST /identity/v3/users => generated 286 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:36:41.699563 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c2ca7-2b87-4a9f-8814-d457a66790a5 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:41.724757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c2ca7-2b87-4a9f-8814-d457a66790a5 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:41.735341 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c853a54-52d0-4e2b-a4ac-ef1a49fc2cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:41.736316 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c853a54-52d0-4e2b-a4ac-ef1a49fc2cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:41.736712 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c853a54-52d0-4e2b-a4ac-ef1a49fc2cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:41.737326 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c853a54-52d0-4e2b-a4ac-ef1a49fc2cf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:41.738249 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c853a54-52d0-4e2b-a4ac-ef1a49fc2cf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:41.740187 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c853a54-52d0-4e2b-a4ac-ef1a49fc2cf9 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:41.771804 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1027/2041] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:41.774622 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ba9bf-df7a-44cc-935c-cd9c274e642d admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:41.814645 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cc74c-6ea4-4557-8d46-da1a4ebe1508 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:41.815098 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cc74c-6ea4-4557-8d46-da1a4ebe1508 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:41.815869 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cc74c-6ea4-4557-8d46-da1a4ebe1508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:41.816293 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ba9bf-df7a-44cc-935c-cd9c274e642d admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:41.831983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b72b1-4b9d-4ef4-89c0-fb1192b708ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:41.831983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b72b1-4b9d-4ef4-89c0-fb1192b708ad admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:41.833259 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b72b1-4b9d-4ef4-89c0-fb1192b708ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:41.833566 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b72b1-4b9d-4ef4-89c0-fb1192b708ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:41.835349 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b72b1-4b9d-4ef4-89c0-fb1192b708ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:41.838423 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b72b1-4b9d-4ef4-89c0-fb1192b708ad admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:41.855898 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1015/2042] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:41.858506 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-cd1cc74c-6ea4-4557-8d46-da1a4ebe1508 None None] MFA Rules not processed for user `b2671aaae84644acb89c3c3595bb5673`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:41.859912 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b72b1-4b9d-4ef4-89c0-fb1192b708ad admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:41.866495 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-cd1cc74c-6ea4-4557-8d46-da1a4ebe1508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:41.881392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b72b1-4b9d-4ef4-89c0-fb1192b708ad admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:41.888911 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d9ea4-7006-4f7c-b4ad-7a4c4e1e70cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:41.890172 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d9ea4-7006-4f7c-b4ad-7a4c4e1e70cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:41.891139 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d9ea4-7006-4f7c-b4ad-7a4c4e1e70cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:41.891656 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d9ea4-7006-4f7c-b4ad-7a4c4e1e70cd admin admin] PATH_INFO: `/v3/projects/54c44b498821414eb37ce58f07274e5d/users/8b9e0ad306d84d039b52d4ccd5f57905/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:41.911907 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d9ea4-7006-4f7c-b4ad-7a4c4e1e70cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=54c44b498821414eb37ce58f07274e5d, user_id=8b9e0ad306d84d039b52d4ccd5f57905, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:41.916631 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d9ea4-7006-4f7c-b4ad-7a4c4e1e70cd admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:41.953459 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1028/2043] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:41.964097 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522f110-66c9-43c0-bc81-f3c5cc04294e None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:41.964438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522f110-66c9-43c0-bc81-f3c5cc04294e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:41.964677 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522f110-66c9-43c0-bc81-f3c5cc04294e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:41.983932 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-d522f110-66c9-43c0-bc81-f3c5cc04294e None None] MFA Rules not processed for user `b2671aaae84644acb89c3c3595bb5673`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:41.985237 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d522f110-66c9-43c0-bc81-f3c5cc04294e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:41.993306 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1029/2044] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:36:42.006523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522f110-66c9-43c0-bc81-f3c5cc04294e None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:42.007961 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d522f110-66c9-43c0-bc81-f3c5cc04294e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:42.033034 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1016/2045] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:36:41 2022] PUT /identity/v3/projects/54c44b498821414eb37ce58f07274e5d/users/8b9e0ad306d84d039b52d4ccd5f57905/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:36:42.046729 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252480f-3b3a-4f9c-a104-fc1db3cd1a69 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:42.046976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252480f-3b3a-4f9c-a104-fc1db3cd1a69 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:42.047368 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252480f-3b3a-4f9c-a104-fc1db3cd1a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:42.086082 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-4252480f-3b3a-4f9c-a104-fc1db3cd1a69 None None] MFA Rules not processed for user `8b9e0ad306d84d039b52d4ccd5f57905`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:42.087431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-4252480f-3b3a-4f9c-a104-fc1db3cd1a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:42.096846 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522f110-66c9-43c0-bc81-f3c5cc04294e None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:42.180904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0f16d-c1fb-4594-9a39-6f45f9d6373c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:42.181668 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0f16d-c1fb-4594-9a39-6f45f9d6373c service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:42.182018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0f16d-c1fb-4594-9a39-6f45f9d6373c service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:42.182362 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0f16d-c1fb-4594-9a39-6f45f9d6373c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:42.187849 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1017/2046] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:42.191464 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252480f-3b3a-4f9c-a104-fc1db3cd1a69 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:42.191464 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-4252480f-3b3a-4f9c-a104-fc1db3cd1a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:42.255004 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c0f16d-c1fb-4594-9a39-6f45f9d6373c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:42.257280 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c0f16d-c1fb-4594-9a39-6f45f9d6373c service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:42.267572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1030/2047] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:36:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:36:42.268658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0f16d-c1fb-4594-9a39-6f45f9d6373c service nova] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:42.282207 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252480f-3b3a-4f9c-a104-fc1db3cd1a69 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:42.290543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe306ca-5e46-4045-a95f-ef5e0dc3ee3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:42.290543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe306ca-5e46-4045-a95f-ef5e0dc3ee3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:42.290543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe306ca-5e46-4045-a95f-ef5e0dc3ee3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:42.290919 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0f16d-c1fb-4594-9a39-6f45f9d6373c service nova] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:42.291003 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe306ca-5e46-4045-a95f-ef5e0dc3ee3d admin admin] PATH_INFO: `/v3/projects/9593b6b2f3b444f2b79dafce83bb484b` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:42.291390 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe306ca-5e46-4045-a95f-ef5e0dc3ee3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9593b6b2f3b444f2b79dafce83bb484b)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:42.293420 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe306ca-5e46-4045-a95f-ef5e0dc3ee3d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:42.294951 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f1316-c491-47e0-9685-0feb21a435da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:42.295823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f1316-c491-47e0-9685-0feb21a435da admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:42.296113 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f1316-c491-47e0-9685-0feb21a435da admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:42.296391 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f1316-c491-47e0-9685-0feb21a435da admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:42.297183 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3f1316-c491-47e0-9685-0feb21a435da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:42.299067 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3f1316-c491-47e0-9685-0feb21a435da admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:42.429218 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1031/2048] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:36:42 2022] POST /identity/v3/projects => generated 373 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:36:42.435803 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f1316-c491-47e0-9685-0feb21a435da admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:42.463930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-cfe306ca-5e46-4045-a95f-ef5e0dc3ee3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9593b6b2f3b444f2b79dafce83bb484b', 'request_id': 'req-cfe306ca-5e46-4045-a95f-ef5e0dc3ee3d'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:42.509925 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f1316-c491-47e0-9685-0feb21a435da admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:42.541107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-cfe306ca-5e46-4045-a95f-ef5e0dc3ee3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9593b6b2f3b444f2b79dafce83bb484b', 'request_id': 'req-cfe306ca-5e46-4045-a95f-ef5e0dc3ee3d'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:42.551708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-cfe306ca-5e46-4045-a95f-ef5e0dc3ee3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9593b6b2f3b444f2b79dafce83bb484b', 'request_id': 'req-cfe306ca-5e46-4045-a95f-ef5e0dc3ee3d'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:42.560726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-cfe306ca-5e46-4045-a95f-ef5e0dc3ee3d admin admin] The token cache is being invalidate because project 9593b6b2f3b444f2b79dafce83bb484b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:36:42.561010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-cfe306ca-5e46-4045-a95f-ef5e0dc3ee3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:42.561680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-cfe306ca-5e46-4045-a95f-ef5e0dc3ee3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:42.563309 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1018/2049] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:36:42 2022] DELETE /identity/v3/projects/9593b6b2f3b444f2b79dafce83bb484b => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:42.564446 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe306ca-5e46-4045-a95f-ef5e0dc3ee3d admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:42.565741 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-cfe306ca-5e46-4045-a95f-ef5e0dc3ee3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:42.580596 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966760b7-adc2-47df-97a9-8767ea23e0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:42.581346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966760b7-adc2-47df-97a9-8767ea23e0c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:42.581571 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966760b7-adc2-47df-97a9-8767ea23e0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:42.581849 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966760b7-adc2-47df-97a9-8767ea23e0c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:42.582552 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966760b7-adc2-47df-97a9-8767ea23e0c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:42.584636 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966760b7-adc2-47df-97a9-8767ea23e0c3 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:42.633487 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:42.636262 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe306ca-5e46-4045-a95f-ef5e0dc3ee3d admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:42.641345 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b588b8-2b82-4a9d-aa10-8e0a831ec773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:42.644242 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b588b8-2b82-4a9d-aa10-8e0a831ec773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:42.644242 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b588b8-2b82-4a9d-aa10-8e0a831ec773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:42.644242 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b588b8-2b82-4a9d-aa10-8e0a831ec773 admin admin] PATH_INFO: `/v3/users/6f4ffcc57da54eb4bafb6382965aa901` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:42.645027 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b588b8-2b82-4a9d-aa10-8e0a831ec773 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f4ffcc57da54eb4bafb6382965aa901)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:42.647064 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b588b8-2b82-4a9d-aa10-8e0a831ec773 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:42.697377 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1032/2050] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:36:42 2022] POST /identity/v3/users => generated 293 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:36:42.698687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966760b7-adc2-47df-97a9-8767ea23e0c3 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:42.701492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-966760b7-adc2-47df-97a9-8767ea23e0c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:42.715876 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966760b7-adc2-47df-97a9-8767ea23e0c3 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:42.720854 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156e1c2-73a8-4df6-9303-c02ca19aaef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:42.723957 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156e1c2-73a8-4df6-9303-c02ca19aaef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:42.723957 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156e1c2-73a8-4df6-9303-c02ca19aaef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:42.723957 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156e1c2-73a8-4df6-9303-c02ca19aaef3 admin admin] PATH_INFO: `/v3/users/2389e99809bf4d8fa534526c2ce49722` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:42.723957 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156e1c2-73a8-4df6-9303-c02ca19aaef3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2389e99809bf4d8fa534526c2ce49722)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:42.726461 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156e1c2-73a8-4df6-9303-c02ca19aaef3 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:42.895534 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-71b588b8-2b82-4a9d-aa10-8e0a831ec773 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f4ffcc57da54eb4bafb6382965aa901'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:42.984608 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4156e1c2-73a8-4df6-9303-c02ca19aaef3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2389e99809bf4d8fa534526c2ce49722'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:42.998373 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-71b588b8-2b82-4a9d-aa10-8e0a831ec773 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f4ffcc57da54eb4bafb6382965aa901'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:42.999163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-71b588b8-2b82-4a9d-aa10-8e0a831ec773 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f4ffcc57da54eb4bafb6382965aa901'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:43.006828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-71b588b8-2b82-4a9d-aa10-8e0a831ec773 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f4ffcc57da54eb4bafb6382965aa901'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:43.008336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-71b588b8-2b82-4a9d-aa10-8e0a831ec773 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f4ffcc57da54eb4bafb6382965aa901'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:43.026773 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1019/2051] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:36:42 2022] DELETE /identity/v3/users/6f4ffcc57da54eb4bafb6382965aa901 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:43.030516 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b588b8-2b82-4a9d-aa10-8e0a831ec773 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:43.035938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-71b588b8-2b82-4a9d-aa10-8e0a831ec773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:43.104320 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b588b8-2b82-4a9d-aa10-8e0a831ec773 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:43.109135 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e33c2f-4359-4dd1-b83d-dc3c4df0aabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:43.109801 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e33c2f-4359-4dd1-b83d-dc3c4df0aabe admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:43.110165 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e33c2f-4359-4dd1-b83d-dc3c4df0aabe admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:43.110433 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e33c2f-4359-4dd1-b83d-dc3c4df0aabe admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:43.111211 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e33c2f-4359-4dd1-b83d-dc3c4df0aabe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:43.112800 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e33c2f-4359-4dd1-b83d-dc3c4df0aabe admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:43.118685 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4156e1c2-73a8-4df6-9303-c02ca19aaef3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2389e99809bf4d8fa534526c2ce49722'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:43.126768 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1020/2052] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:43.126768 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712cadb-930f-45d0-a908-d3911ce387ea None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:43.126768 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712cadb-930f-45d0-a908-d3911ce387ea None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:43.126768 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712cadb-930f-45d0-a908-d3911ce387ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:43.146507 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-c712cadb-930f-45d0-a908-d3911ce387ea None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:43.146749 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4156e1c2-73a8-4df6-9303-c02ca19aaef3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2389e99809bf4d8fa534526c2ce49722'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:43.148031 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4156e1c2-73a8-4df6-9303-c02ca19aaef3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2389e99809bf4d8fa534526c2ce49722'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:43.148150 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c712cadb-930f-45d0-a908-d3911ce387ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:43.155514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4156e1c2-73a8-4df6-9303-c02ca19aaef3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2389e99809bf4d8fa534526c2ce49722'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:43.156345 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1021/2053] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:43.157664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c712cadb-930f-45d0-a908-d3911ce387ea None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:43.165345 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c712cadb-930f-45d0-a908-d3911ce387ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:43.166603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1033/2054] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:36:42 2022] DELETE /identity/v3/users/2389e99809bf4d8fa534526c2ce49722 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:43.167468 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156e1c2-73a8-4df6-9303-c02ca19aaef3 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:43.168847 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4156e1c2-73a8-4df6-9303-c02ca19aaef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:43.177795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c712cadb-930f-45d0-a908-d3911ce387ea None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:43.257108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156e1c2-73a8-4df6-9303-c02ca19aaef3 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:43.263081 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e275816a-4012-4179-b3cf-0eea636ed12a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:43.264166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275816a-4012-4179-b3cf-0eea636ed12a service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:43.268122 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275816a-4012-4179-b3cf-0eea636ed12a service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:43.268122 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275816a-4012-4179-b3cf-0eea636ed12a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:43.276575 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe5383-14e8-40f8-8321-9387d93706b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:43.277497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe5383-14e8-40f8-8321-9387d93706b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:43.277778 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe5383-14e8-40f8-8321-9387d93706b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:43.278050 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe5383-14e8-40f8-8321-9387d93706b7 admin admin] PATH_INFO: `/v3/projects/6ed9d44c9a094bdc892a908110db882f/users/46ed835b11244870952aaec4b2b6f771/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:43.299937 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe5383-14e8-40f8-8321-9387d93706b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ed9d44c9a094bdc892a908110db882f, user_id=46ed835b11244870952aaec4b2b6f771, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:43.303399 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efe5383-14e8-40f8-8321-9387d93706b7 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:43.343049 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e275816a-4012-4179-b3cf-0eea636ed12a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:43.344457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e275816a-4012-4179-b3cf-0eea636ed12a service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:43.354746 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1034/2055] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:36:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:43.432150 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1022/2056] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:36:43 2022] PUT /identity/v3/projects/6ed9d44c9a094bdc892a908110db882f/users/46ed835b11244870952aaec4b2b6f771/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:43.436259 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e275816a-4012-4179-b3cf-0eea636ed12a service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:43.449120 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e275816a-4012-4179-b3cf-0eea636ed12a service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:43.539061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f5710-192e-49b8-9e44-8344e7bf0139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:43.539839 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f5710-192e-49b8-9e44-8344e7bf0139 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:43.540205 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f5710-192e-49b8-9e44-8344e7bf0139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:43.540556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f5710-192e-49b8-9e44-8344e7bf0139 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:43.541366 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76f5710-192e-49b8-9e44-8344e7bf0139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:43.543327 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76f5710-192e-49b8-9e44-8344e7bf0139 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:43.556035 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1035/2057] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:43.561285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe5383-14e8-40f8-8321-9387d93706b7 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:43.576257 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe5383-14e8-40f8-8321-9387d93706b7 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:43.586236 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595be8c-6968-4eea-a746-1d4ff5691c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:43.587332 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595be8c-6968-4eea-a746-1d4ff5691c90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:43.589962 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595be8c-6968-4eea-a746-1d4ff5691c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:43.589962 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595be8c-6968-4eea-a746-1d4ff5691c90 admin admin] PATH_INFO: `/v3/projects/6ed9d44c9a094bdc892a908110db882f/users/46ed835b11244870952aaec4b2b6f771/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:43.590668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595be8c-6968-4eea-a746-1d4ff5691c90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ed9d44c9a094bdc892a908110db882f, user_id=46ed835b11244870952aaec4b2b6f771, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:43.593658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595be8c-6968-4eea-a746-1d4ff5691c90 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:43.607333 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1023/2058] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:36:43 2022] PUT /identity/v3/projects/6ed9d44c9a094bdc892a908110db882f/users/46ed835b11244870952aaec4b2b6f771/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:43.623384 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155ef56-d591-4b7a-8ee6-1189d14965aa None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:43.623384 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155ef56-d591-4b7a-8ee6-1189d14965aa None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:43.623384 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155ef56-d591-4b7a-8ee6-1189d14965aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:43.662490 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-2155ef56-d591-4b7a-8ee6-1189d14965aa None None] MFA Rules not processed for user `46ed835b11244870952aaec4b2b6f771`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:43.663917 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-2155ef56-d591-4b7a-8ee6-1189d14965aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:43.736135 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:43.785136 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1036/2059] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:43 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:43.796249 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595be8c-6968-4eea-a746-1d4ff5691c90 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:43.798083 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c595be8c-6968-4eea-a746-1d4ff5691c90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:43.812238 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2155ef56-d591-4b7a-8ee6-1189d14965aa None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:43.813885 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-2155ef56-d591-4b7a-8ee6-1189d14965aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:43.862124 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595be8c-6968-4eea-a746-1d4ff5691c90 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:43.867298 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0113dc-dcb1-49d9-ad0d-cdb9b1424bc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:43.870281 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0113dc-dcb1-49d9-ad0d-cdb9b1424bc0 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:43.870281 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0113dc-dcb1-49d9-ad0d-cdb9b1424bc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:43.870281 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0113dc-dcb1-49d9-ad0d-cdb9b1424bc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:43.879278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0113dc-dcb1-49d9-ad0d-cdb9b1424bc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:43.880931 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0113dc-dcb1-49d9-ad0d-cdb9b1424bc0 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:43.891650 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1024/2060] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:36:43 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:43.901541 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2155ef56-d591-4b7a-8ee6-1189d14965aa None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:43.910088 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0113dc-dcb1-49d9-ad0d-cdb9b1424bc0 service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:43.913431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-af0113dc-dcb1-49d9-ad0d-cdb9b1424bc0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:43.929996 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0113dc-dcb1-49d9-ad0d-cdb9b1424bc0 service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:43.939195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35820288-099f-4a26-86f5-dfe3146a006e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:43.940214 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35820288-099f-4a26-86f5-dfe3146a006e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:43.940331 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35820288-099f-4a26-86f5-dfe3146a006e admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:43.940605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35820288-099f-4a26-86f5-dfe3146a006e admin admin] PATH_INFO: `/v3/projects/b3424b242afe48229284fe05a4ce04ed` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:43.941394 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f9417-dc54-4fb8-8e7c-7ef36ff91c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:43.944019 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f9417-dc54-4fb8-8e7c-7ef36ff91c13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:43.944019 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f9417-dc54-4fb8-8e7c-7ef36ff91c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:43.944447 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f9417-dc54-4fb8-8e7c-7ef36ff91c13 admin admin] PATH_INFO: `/v3/projects/c775c084721e4a6187b84080a70ebe8d` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:43.945710 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35820288-099f-4a26-86f5-dfe3146a006e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3424b242afe48229284fe05a4ce04ed)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:43.946495 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f9417-dc54-4fb8-8e7c-7ef36ff91c13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c775c084721e4a6187b84080a70ebe8d)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:43.949684 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35820288-099f-4a26-86f5-dfe3146a006e admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:43.949906 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f9417-dc54-4fb8-8e7c-7ef36ff91c13 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:44.140920 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-35820288-099f-4a26-86f5-dfe3146a006e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3424b242afe48229284fe05a4ce04ed', 'request_id': 'req-35820288-099f-4a26-86f5-dfe3146a006e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:44.155643 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ec5f9417-dc54-4fb8-8e7c-7ef36ff91c13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c775c084721e4a6187b84080a70ebe8d', 'request_id': 'req-ec5f9417-dc54-4fb8-8e7c-7ef36ff91c13'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:44.266813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-35820288-099f-4a26-86f5-dfe3146a006e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3424b242afe48229284fe05a4ce04ed', 'request_id': 'req-35820288-099f-4a26-86f5-dfe3146a006e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:44.278870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-35820288-099f-4a26-86f5-dfe3146a006e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3424b242afe48229284fe05a4ce04ed', 'request_id': 'req-35820288-099f-4a26-86f5-dfe3146a006e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:44.291713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-35820288-099f-4a26-86f5-dfe3146a006e admin admin] The token cache is being invalidate because project b3424b242afe48229284fe05a4ce04ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:36:44.292004 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-35820288-099f-4a26-86f5-dfe3146a006e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:44.292683 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-35820288-099f-4a26-86f5-dfe3146a006e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:44.294595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1025/2061] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:36:43 2022] DELETE /identity/v3/projects/b3424b242afe48229284fe05a4ce04ed => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:44.300985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd9f94-7caa-4d00-bda3-3ae8c57c92d7 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:44.301816 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd9f94-7caa-4d00-bda3-3ae8c57c92d7 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:44.301816 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd9f94-7caa-4d00-bda3-3ae8c57c92d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:44.366315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-ffcd9f94-7caa-4d00-bda3-3ae8c57c92d7 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:44.367355 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ffcd9f94-7caa-4d00-bda3-3ae8c57c92d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:44.389718 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ec5f9417-dc54-4fb8-8e7c-7ef36ff91c13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c775c084721e4a6187b84080a70ebe8d', 'request_id': 'req-ec5f9417-dc54-4fb8-8e7c-7ef36ff91c13'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:44.401105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ec5f9417-dc54-4fb8-8e7c-7ef36ff91c13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c775c084721e4a6187b84080a70ebe8d', 'request_id': 'req-ec5f9417-dc54-4fb8-8e7c-7ef36ff91c13'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:44.417156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ec5f9417-dc54-4fb8-8e7c-7ef36ff91c13 admin admin] The token cache is being invalidate because project c775c084721e4a6187b84080a70ebe8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:36:44.417626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ec5f9417-dc54-4fb8-8e7c-7ef36ff91c13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:44.418930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ec5f9417-dc54-4fb8-8e7c-7ef36ff91c13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:44.420796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1037/2062] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:36:43 2022] DELETE /identity/v3/projects/c775c084721e4a6187b84080a70ebe8d => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:36:44.434080 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f9379-34a0-4ea2-a64e-686d2f1422d3 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:44.434659 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f9379-34a0-4ea2-a64e-686d2f1422d3 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:44.435067 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f9379-34a0-4ea2-a64e-686d2f1422d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:44.482240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-468f9379-34a0-4ea2-a64e-686d2f1422d3 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:44.483733 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-468f9379-34a0-4ea2-a64e-686d2f1422d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:44.507997 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1026/2063] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:36:44.517112 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff372585-4073-4e33-b92e-90d169aab5ba None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:44.517633 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff372585-4073-4e33-b92e-90d169aab5ba None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:44.518060 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff372585-4073-4e33-b92e-90d169aab5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:44.544158 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-ff372585-4073-4e33-b92e-90d169aab5ba None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:44.545674 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ff372585-4073-4e33-b92e-90d169aab5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:44.556678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1027/2064] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:44.562184 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff372585-4073-4e33-b92e-90d169aab5ba None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:44.573778 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff372585-4073-4e33-b92e-90d169aab5ba None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:44.577943 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1038/2065] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:44.587116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f51d4d-aabb-4a18-b6d7-532cadb62258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:44.588839 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f51d4d-aabb-4a18-b6d7-532cadb62258 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:44.588839 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f51d4d-aabb-4a18-b6d7-532cadb62258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:44.589256 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f51d4d-aabb-4a18-b6d7-532cadb62258 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:44.590371 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f51d4d-aabb-4a18-b6d7-532cadb62258 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:36:44.590933 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f51d4d-aabb-4a18-b6d7-532cadb62258 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:44.593182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f51d4d-aabb-4a18-b6d7-532cadb62258 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:44.598642 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40ba75-6616-4d35-a7c2-21d3656beeaa None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:44.600905 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40ba75-6616-4d35-a7c2-21d3656beeaa None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:44.601236 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40ba75-6616-4d35-a7c2-21d3656beeaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:44.633676 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1028/2066] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:36:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:36:44.643306 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f51d4d-aabb-4a18-b6d7-532cadb62258 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:44.655747 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f51d4d-aabb-4a18-b6d7-532cadb62258 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:44.657111 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-0f40ba75-6616-4d35-a7c2-21d3656beeaa None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:44.661235 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-0f40ba75-6616-4d35-a7c2-21d3656beeaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:44.670905 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcaf865-7208-439e-95f5-1e7d86958579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:44.671702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcaf865-7208-439e-95f5-1e7d86958579 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:44.672108 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcaf865-7208-439e-95f5-1e7d86958579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:44.672523 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcaf865-7208-439e-95f5-1e7d86958579 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:44.673383 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcaf865-7208-439e-95f5-1e7d86958579 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:44.675665 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcaf865-7208-439e-95f5-1e7d86958579 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:44.677099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1039/2067] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:44.684350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40ba75-6616-4d35-a7c2-21d3656beeaa None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:44.701195 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40ba75-6616-4d35-a7c2-21d3656beeaa None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:44.707566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ee57e-2fb6-444f-8c5e-392f3ec14f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:44.708593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ee57e-2fb6-444f-8c5e-392f3ec14f26 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:44.710421 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ee57e-2fb6-444f-8c5e-392f3ec14f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:44.710776 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ee57e-2fb6-444f-8c5e-392f3ec14f26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:44.711716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ee57e-2fb6-444f-8c5e-392f3ec14f26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:36:44.712081 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ee57e-2fb6-444f-8c5e-392f3ec14f26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:44.713797 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ee57e-2fb6-444f-8c5e-392f3ec14f26 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:44.734908 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1040/2068] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:36:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:36:44.739286 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ee57e-2fb6-444f-8c5e-392f3ec14f26 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:44.754025 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ee57e-2fb6-444f-8c5e-392f3ec14f26 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:44.759751 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e59b1d-f0c0-45d9-83f7-d38ab6491e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:44.760404 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e59b1d-f0c0-45d9-83f7-d38ab6491e2b admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:44.763763 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e59b1d-f0c0-45d9-83f7-d38ab6491e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:44.763763 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e59b1d-f0c0-45d9-83f7-d38ab6491e2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:44.763763 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e59b1d-f0c0-45d9-83f7-d38ab6491e2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:44.764125 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e59b1d-f0c0-45d9-83f7-d38ab6491e2b admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:44.853564 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1029/2069] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:36:44 2022] POST /identity/v3/projects => generated 371 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:36:44.859220 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcaf865-7208-439e-95f5-1e7d86958579 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:44.886040 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1041/2070] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:36:44 2022] POST /identity/v3/projects => generated 367 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:36:44.891304 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e59b1d-f0c0-45d9-83f7-d38ab6491e2b admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:44.955589 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcaf865-7208-439e-95f5-1e7d86958579 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:45.005600 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e59b1d-f0c0-45d9-83f7-d38ab6491e2b admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:45.012371 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc0312-d72c-4f83-8b0a-e5ccb14d262b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:45.013175 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc0312-d72c-4f83-8b0a-e5ccb14d262b admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:45.013514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc0312-d72c-4f83-8b0a-e5ccb14d262b admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:45.014444 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc0312-d72c-4f83-8b0a-e5ccb14d262b admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:45.015247 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc0312-d72c-4f83-8b0a-e5ccb14d262b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:45.024071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc0312-d72c-4f83-8b0a-e5ccb14d262b admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:45.113273 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5777e9a7-c68d-4830-aa02-77460262e5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:45.114089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5777e9a7-c68d-4830-aa02-77460262e5b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:45.119501 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5777e9a7-c68d-4830-aa02-77460262e5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:45.119501 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5777e9a7-c68d-4830-aa02-77460262e5b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:45.119501 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5777e9a7-c68d-4830-aa02-77460262e5b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:45.119501 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5777e9a7-c68d-4830-aa02-77460262e5b0 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:45.137006 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1042/2071] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:36:44 2022] POST /identity/v3/users => generated 284 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:36:45.141346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc0312-d72c-4f83-8b0a-e5ccb14d262b admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:45.153899 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc0312-d72c-4f83-8b0a-e5ccb14d262b admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:45.158124 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078fd5ad-4127-43ce-8dd3-b050c47238bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:45.158798 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078fd5ad-4127-43ce-8dd3-b050c47238bf admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:45.159115 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078fd5ad-4127-43ce-8dd3-b050c47238bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:45.159388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078fd5ad-4127-43ce-8dd3-b050c47238bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:45.160151 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078fd5ad-4127-43ce-8dd3-b050c47238bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:45.161886 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078fd5ad-4127-43ce-8dd3-b050c47238bf admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:45.176349 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1043/2072] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:45.177398 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078fd5ad-4127-43ce-8dd3-b050c47238bf admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:45.190447 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078fd5ad-4127-43ce-8dd3-b050c47238bf admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:45.198559 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb4442-b55e-4b15-be9a-f4f1564b91c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:45.199476 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb4442-b55e-4b15-be9a-f4f1564b91c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:45.199756 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb4442-b55e-4b15-be9a-f4f1564b91c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:45.210610 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb4442-b55e-4b15-be9a-f4f1564b91c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:45.210610 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb4442-b55e-4b15-be9a-f4f1564b91c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:45.210610 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb4442-b55e-4b15-be9a-f4f1564b91c7 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:45.219048 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1044/2073] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:36:45.223170 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb4442-b55e-4b15-be9a-f4f1564b91c7 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:45.233349 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb4442-b55e-4b15-be9a-f4f1564b91c7 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:45.241343 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3652f-7da8-4f6c-8b0c-1e8dc9d29e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:45.242267 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3652f-7da8-4f6c-8b0c-1e8dc9d29e4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:45.244800 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3652f-7da8-4f6c-8b0c-1e8dc9d29e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:45.244800 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3652f-7da8-4f6c-8b0c-1e8dc9d29e4c admin admin] PATH_INFO: `/v3/projects/ed4021fb1d7e465d8439f72923f5d651/users/1b3ff9487a154831afa6ae492a0b0f7c/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:45.260056 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b3652f-7da8-4f6c-8b0c-1e8dc9d29e4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed4021fb1d7e465d8439f72923f5d651, user_id=1b3ff9487a154831afa6ae492a0b0f7c, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:45.263884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b3652f-7da8-4f6c-8b0c-1e8dc9d29e4c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:45.316907 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1030/2074] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:36:44 2022] POST /identity/v3/users => generated 286 bytes in 459 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:36:45.321627 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5777e9a7-c68d-4830-aa02-77460262e5b0 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:45.340071 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5777e9a7-c68d-4830-aa02-77460262e5b0 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:45.341700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5208cc2-37f3-4796-acf4-6b3efd7a4c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:45.342428 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5208cc2-37f3-4796-acf4-6b3efd7a4c8b admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:45.342647 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5208cc2-37f3-4796-acf4-6b3efd7a4c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:45.342947 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5208cc2-37f3-4796-acf4-6b3efd7a4c8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:45.343732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5208cc2-37f3-4796-acf4-6b3efd7a4c8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:45.345411 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5208cc2-37f3-4796-acf4-6b3efd7a4c8b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:45.367884 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1031/2075] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:36:45.371731 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5208cc2-37f3-4796-acf4-6b3efd7a4c8b admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:45.388572 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5208cc2-37f3-4796-acf4-6b3efd7a4c8b admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:45.396765 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d058f7fa-4ef5-4c4b-a935-438d0b434aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:45.397600 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058f7fa-4ef5-4c4b-a935-438d0b434aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:45.397898 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058f7fa-4ef5-4c4b-a935-438d0b434aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:45.398177 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058f7fa-4ef5-4c4b-a935-438d0b434aa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:45.398944 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d058f7fa-4ef5-4c4b-a935-438d0b434aa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:45.404516 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d058f7fa-4ef5-4c4b-a935-438d0b434aa7 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:45.413775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1032/2076] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:36:45.417801 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d058f7fa-4ef5-4c4b-a935-438d0b434aa7 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:45.427553 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:45.438198 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d058f7fa-4ef5-4c4b-a935-438d0b434aa7 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:45.449027 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1045/2077] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:36:45 2022] PUT /identity/v3/projects/ed4021fb1d7e465d8439f72923f5d651/users/1b3ff9487a154831afa6ae492a0b0f7c/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:36:45.449027 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3652f-7da8-4f6c-8b0c-1e8dc9d29e4c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:45.449287 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-47b3652f-7da8-4f6c-8b0c-1e8dc9d29e4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:45.507045 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71365050-54d8-4fa4-8f2a-2465347d071e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:45.507949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71365050-54d8-4fa4-8f2a-2465347d071e admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:45.508442 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71365050-54d8-4fa4-8f2a-2465347d071e admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:45.508750 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71365050-54d8-4fa4-8f2a-2465347d071e admin admin] PATH_INFO: `/v3/projects/19592ab2e5e04d7c952aebf73f4fca6e/users/9001a4b10d544226b3b1453b06883db4/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:45.531091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71365050-54d8-4fa4-8f2a-2465347d071e admin admin] RBAC: Authorizing `identity:create_grant(project_id=19592ab2e5e04d7c952aebf73f4fca6e, user_id=9001a4b10d544226b3b1453b06883db4, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:45.533399 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3652f-7da8-4f6c-8b0c-1e8dc9d29e4c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:45.533920 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71365050-54d8-4fa4-8f2a-2465347d071e admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:45.538874 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7130ca-834c-46b7-bbd1-bc293d8b25f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:45.539759 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7130ca-834c-46b7-bbd1-bc293d8b25f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:45.540035 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7130ca-834c-46b7-bbd1-bc293d8b25f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:45.540306 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7130ca-834c-46b7-bbd1-bc293d8b25f8 admin admin] PATH_INFO: `/v3/users/8b9e0ad306d84d039b52d4ccd5f57905` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:45.542400 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7130ca-834c-46b7-bbd1-bc293d8b25f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b9e0ad306d84d039b52d4ccd5f57905)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:45.543490 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7130ca-834c-46b7-bbd1-bc293d8b25f8 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:45.602981 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:45.639285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1033/2078] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:36:45 2022] PUT /identity/v3/projects/19592ab2e5e04d7c952aebf73f4fca6e/users/9001a4b10d544226b3b1453b06883db4/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:45.641079 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796f69b-8af7-408f-9774-0ed5396874ea None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:45.641374 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796f69b-8af7-408f-9774-0ed5396874ea None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:45.641652 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796f69b-8af7-408f-9774-0ed5396874ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:45.671133 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-0796f69b-8af7-408f-9774-0ed5396874ea None None] MFA Rules not processed for user `1b3ff9487a154831afa6ae492a0b0f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:45.672348 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-0796f69b-8af7-408f-9774-0ed5396874ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:45.730821 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bc7130ca-834c-46b7-bbd1-bc293d8b25f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b9e0ad306d84d039b52d4ccd5f57905'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:45.747197 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1034/2079] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:45 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:45.749592 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470851d9-31ac-45d0-85cc-a9a1966974f0 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:45.750191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470851d9-31ac-45d0-85cc-a9a1966974f0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:45.750863 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470851d9-31ac-45d0-85cc-a9a1966974f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:45.774903 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-470851d9-31ac-45d0-85cc-a9a1966974f0 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:45.776243 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-470851d9-31ac-45d0-85cc-a9a1966974f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:45.808159 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bc7130ca-834c-46b7-bbd1-bc293d8b25f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b9e0ad306d84d039b52d4ccd5f57905'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:45.819673 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bc7130ca-834c-46b7-bbd1-bc293d8b25f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b9e0ad306d84d039b52d4ccd5f57905'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:45.821255 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bc7130ca-834c-46b7-bbd1-bc293d8b25f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b9e0ad306d84d039b52d4ccd5f57905'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:45.830041 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-bc7130ca-834c-46b7-bbd1-bc293d8b25f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b9e0ad306d84d039b52d4ccd5f57905'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:45.837792 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1046/2080] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:36:45 2022] DELETE /identity/v3/users/8b9e0ad306d84d039b52d4ccd5f57905 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:45.838622 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7130ca-834c-46b7-bbd1-bc293d8b25f8 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:45.839992 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-bc7130ca-834c-46b7-bbd1-bc293d8b25f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:45.849006 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1035/2081] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:45.851253 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b00d63-5035-40b3-9922-746daaf1f226 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:45.851611 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b00d63-5035-40b3-9922-746daaf1f226 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:45.851958 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b00d63-5035-40b3-9922-746daaf1f226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:45.875935 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d6b00d63-5035-40b3-9922-746daaf1f226 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:45.877195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d6b00d63-5035-40b3-9922-746daaf1f226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:45.926663 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7130ca-834c-46b7-bbd1-bc293d8b25f8 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:45.932363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f411d3-6a8b-42aa-afb6-c56f40740d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:45.935090 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f411d3-6a8b-42aa-afb6-c56f40740d5a admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:45.939060 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f411d3-6a8b-42aa-afb6-c56f40740d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:45.939060 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f411d3-6a8b-42aa-afb6-c56f40740d5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:45.939060 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f411d3-6a8b-42aa-afb6-c56f40740d5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:45.939575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f411d3-6a8b-42aa-afb6-c56f40740d5a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:45.953454 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1036/2082] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:45.958744 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b00d63-5035-40b3-9922-746daaf1f226 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:45.964878 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d6b00d63-5035-40b3-9922-746daaf1f226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:46.018126 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b00d63-5035-40b3-9922-746daaf1f226 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:46.022590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc66a7-a51e-47b4-bf26-3b2a39e38bea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:46.023167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc66a7-a51e-47b4-bf26-3b2a39e38bea service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:46.023451 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc66a7-a51e-47b4-bf26-3b2a39e38bea service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:46.023723 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc66a7-a51e-47b4-bf26-3b2a39e38bea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:46.034146 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc66a7-a51e-47b4-bf26-3b2a39e38bea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:46.035825 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc66a7-a51e-47b4-bf26-3b2a39e38bea service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:46.052185 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1047/2083] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:36:45 2022] POST /identity/v3/projects => generated 365 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:36:46.053397 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f411d3-6a8b-42aa-afb6-c56f40740d5a admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:46.055372 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-64f411d3-6a8b-42aa-afb6-c56f40740d5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:46.061273 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1037/2084] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:36:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:46.061814 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc66a7-a51e-47b4-bf26-3b2a39e38bea service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:46.082481 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc66a7-a51e-47b4-bf26-3b2a39e38bea service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:46.173804 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f411d3-6a8b-42aa-afb6-c56f40740d5a admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:46.184987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f02648-fa52-407d-9247-19f6b734975e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:46.185234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a63b7-d18b-4f64-b7cd-cc1cc450f91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:46.185655 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f02648-fa52-407d-9247-19f6b734975e service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:46.185881 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a63b7-d18b-4f64-b7cd-cc1cc450f91d admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:46.185980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f02648-fa52-407d-9247-19f6b734975e service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:46.186164 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a63b7-d18b-4f64-b7cd-cc1cc450f91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:46.186263 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f02648-fa52-407d-9247-19f6b734975e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:46.186437 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a63b7-d18b-4f64-b7cd-cc1cc450f91d admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:46.187174 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a63b7-d18b-4f64-b7cd-cc1cc450f91d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:46.189409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a63b7-d18b-4f64-b7cd-cc1cc450f91d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:46.216892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f02648-fa52-407d-9247-19f6b734975e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:46.218429 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f02648-fa52-407d-9247-19f6b734975e service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:46.243282 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1048/2085] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:36:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:46.329357 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1038/2086] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:36:46 2022] POST /identity/v3/users => generated 289 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:36:46.330160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f02648-fa52-407d-9247-19f6b734975e service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:46.356756 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f02648-fa52-407d-9247-19f6b734975e service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:46.378029 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb028f3e-33c8-474a-9656-714dfb009722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:46.379410 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb028f3e-33c8-474a-9656-714dfb009722 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:46.379885 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb028f3e-33c8-474a-9656-714dfb009722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:46.380345 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb028f3e-33c8-474a-9656-714dfb009722 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:46.382011 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb028f3e-33c8-474a-9656-714dfb009722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:46.385329 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb028f3e-33c8-474a-9656-714dfb009722 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:46.414890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1049/2087] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:36:46.418065 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a63b7-d18b-4f64-b7cd-cc1cc450f91d admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:46.434339 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a63b7-d18b-4f64-b7cd-cc1cc450f91d admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:46.440999 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822398c1-0f4e-4f0e-a177-d2a78463a482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:46.441930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822398c1-0f4e-4f0e-a177-d2a78463a482 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:46.442207 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822398c1-0f4e-4f0e-a177-d2a78463a482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:46.442500 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822398c1-0f4e-4f0e-a177-d2a78463a482 admin admin] PATH_INFO: `/v3/projects/6ce9429917ee47fa94232afb35703316/users/d529f0143eb74946a719e301cab5fe43/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:46.454101 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:46.463774 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822398c1-0f4e-4f0e-a177-d2a78463a482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ce9429917ee47fa94232afb35703316, user_id=d529f0143eb74946a719e301cab5fe43, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:46.466331 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822398c1-0f4e-4f0e-a177-d2a78463a482 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:46.592161 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1039/2088] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:36:46 2022] PUT /identity/v3/projects/6ce9429917ee47fa94232afb35703316/users/d529f0143eb74946a719e301cab5fe43/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:46.595995 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb028f3e-33c8-474a-9656-714dfb009722 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:46.613839 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb028f3e-33c8-474a-9656-714dfb009722 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:46.747089 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcde76d-5fb4-4a51-a609-a84e30e38afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:46.747964 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcde76d-5fb4-4a51-a609-a84e30e38afd admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:46.748242 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcde76d-5fb4-4a51-a609-a84e30e38afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:46.748520 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcde76d-5fb4-4a51-a609-a84e30e38afd admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:46.749283 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcde76d-5fb4-4a51-a609-a84e30e38afd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:46.751276 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcde76d-5fb4-4a51-a609-a84e30e38afd admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:46.764908 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1050/2089] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:46.769185 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822398c1-0f4e-4f0e-a177-d2a78463a482 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:46.781417 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822398c1-0f4e-4f0e-a177-d2a78463a482 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:46.796006 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e8f77-74d2-47c5-b434-e6b813741b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:46.799518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e8f77-74d2-47c5-b434-e6b813741b97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:46.799518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e8f77-74d2-47c5-b434-e6b813741b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:46.799518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e8f77-74d2-47c5-b434-e6b813741b97 admin admin] PATH_INFO: `/v3/projects/6ce9429917ee47fa94232afb35703316/users/d529f0143eb74946a719e301cab5fe43/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:46.799518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7e8f77-74d2-47c5-b434-e6b813741b97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ce9429917ee47fa94232afb35703316, user_id=d529f0143eb74946a719e301cab5fe43, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:46.802387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7e8f77-74d2-47c5-b434-e6b813741b97 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:46.814304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1040/2090] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:36:46 2022] PUT /identity/v3/projects/6ce9429917ee47fa94232afb35703316/users/d529f0143eb74946a719e301cab5fe43/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:46.824504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ff4ff-ccd7-4891-899e-3c1b372fb812 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:46.824918 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ff4ff-ccd7-4891-899e-3c1b372fb812 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:46.825307 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ff4ff-ccd7-4891-899e-3c1b372fb812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:46.882065 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-3f1ff4ff-ccd7-4891-899e-3c1b372fb812 None None] MFA Rules not processed for user `d529f0143eb74946a719e301cab5fe43`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:46.883645 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3f1ff4ff-ccd7-4891-899e-3c1b372fb812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:46.993412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1051/2091] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:46 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:36:47.005693 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e8f77-74d2-47c5-b434-e6b813741b97 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:47.008483 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-5c7e8f77-74d2-47c5-b434-e6b813741b97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:47.025861 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ff4ff-ccd7-4891-899e-3c1b372fb812 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:47.033076 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3f1ff4ff-ccd7-4891-899e-3c1b372fb812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:47.132608 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7e8f77-74d2-47c5-b434-e6b813741b97 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:47.133816 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ff4ff-ccd7-4891-899e-3c1b372fb812 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:47.137891 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f3920-ecd8-4d6b-997a-a82066da0fb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:47.138956 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f3920-ecd8-4d6b-997a-a82066da0fb1 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:47.139200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5efedb-ea9e-4da0-bba3-34f455b42f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:47.140091 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5efedb-ea9e-4da0-bba3-34f455b42f1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:47.140326 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f3920-ecd8-4d6b-997a-a82066da0fb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:47.140458 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5efedb-ea9e-4da0-bba3-34f455b42f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:47.140636 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5efedb-ea9e-4da0-bba3-34f455b42f1c admin admin] PATH_INFO: `/v3/projects/54c44b498821414eb37ce58f07274e5d` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:47.140936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f3920-ecd8-4d6b-997a-a82066da0fb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:47.142111 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5efedb-ea9e-4da0-bba3-34f455b42f1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=54c44b498821414eb37ce58f07274e5d)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:47.148690 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5efedb-ea9e-4da0-bba3-34f455b42f1c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:47.154195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f3920-ecd8-4d6b-997a-a82066da0fb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:47.155878 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f3920-ecd8-4d6b-997a-a82066da0fb1 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:47.170605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1041/2092] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:36:47 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:47.346466 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-cd5efedb-ea9e-4da0-bba3-34f455b42f1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54c44b498821414eb37ce58f07274e5d', 'request_id': 'req-cd5efedb-ea9e-4da0-bba3-34f455b42f1c'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:47.441415 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-cd5efedb-ea9e-4da0-bba3-34f455b42f1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54c44b498821414eb37ce58f07274e5d', 'request_id': 'req-cd5efedb-ea9e-4da0-bba3-34f455b42f1c'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:47.453872 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-cd5efedb-ea9e-4da0-bba3-34f455b42f1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54c44b498821414eb37ce58f07274e5d', 'request_id': 'req-cd5efedb-ea9e-4da0-bba3-34f455b42f1c'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:47.462863 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d67eec-8eb1-4f2f-af8f-cc1fa56c3a4d None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:47.463182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d67eec-8eb1-4f2f-af8f-cc1fa56c3a4d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:47.463462 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d67eec-8eb1-4f2f-af8f-cc1fa56c3a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:47.478183 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-cd5efedb-ea9e-4da0-bba3-34f455b42f1c admin admin] The token cache is being invalidate because project 54c44b498821414eb37ce58f07274e5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:36:47.478183 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-cd5efedb-ea9e-4da0-bba3-34f455b42f1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:47.478183 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-cd5efedb-ea9e-4da0-bba3-34f455b42f1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:47.480307 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1052/2093] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:36:47 2022] DELETE /identity/v3/projects/54c44b498821414eb37ce58f07274e5d => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:36:47.482424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5efedb-ea9e-4da0-bba3-34f455b42f1c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:47.483991 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-cd5efedb-ea9e-4da0-bba3-34f455b42f1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:47.505515 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-61d67eec-8eb1-4f2f-af8f-cc1fa56c3a4d None None] MFA Rules not processed for user `b2671aaae84644acb89c3c3595bb5673`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:47.508006 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-61d67eec-8eb1-4f2f-af8f-cc1fa56c3a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:47.575388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5efedb-ea9e-4da0-bba3-34f455b42f1c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:47.579581 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfe2e38-ae5f-406b-8501-9867a0e37374 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:47.580232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe2e38-ae5f-406b-8501-9867a0e37374 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:47.580615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe2e38-ae5f-406b-8501-9867a0e37374 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:47.580795 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfe2e38-ae5f-406b-8501-9867a0e37374 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:47.583672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3cfe2e38-ae5f-406b-8501-9867a0e37374 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:47.623270 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1042/2094] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:47.624211 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d67eec-8eb1-4f2f-af8f-cc1fa56c3a4d None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:47.628136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-61d67eec-8eb1-4f2f-af8f-cc1fa56c3a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:47.669828 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfe2e38-ae5f-406b-8501-9867a0e37374 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:47.671333 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfe2e38-ae5f-406b-8501-9867a0e37374 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:47.678568 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:47.684938 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1053/2095] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:36:47 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:47.757683 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d67eec-8eb1-4f2f-af8f-cc1fa56c3a4d None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:47.770579 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255b069-5d67-4d00-9a5f-99b7d1780890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:47.782004 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255b069-5d67-4d00-9a5f-99b7d1780890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:47.782004 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255b069-5d67-4d00-9a5f-99b7d1780890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:47.782004 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255b069-5d67-4d00-9a5f-99b7d1780890 admin admin] PATH_INFO: `/v3/users/46ed835b11244870952aaec4b2b6f771` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:47.782004 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d255b069-5d67-4d00-9a5f-99b7d1780890 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46ed835b11244870952aaec4b2b6f771)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:47.782004 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d255b069-5d67-4d00-9a5f-99b7d1780890 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:48.033968 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d255b069-5d67-4d00-9a5f-99b7d1780890 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46ed835b11244870952aaec4b2b6f771'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:48.152606 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d255b069-5d67-4d00-9a5f-99b7d1780890 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46ed835b11244870952aaec4b2b6f771'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:48.153285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d255b069-5d67-4d00-9a5f-99b7d1780890 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46ed835b11244870952aaec4b2b6f771'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:48.164639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d255b069-5d67-4d00-9a5f-99b7d1780890 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46ed835b11244870952aaec4b2b6f771'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:48.165678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d255b069-5d67-4d00-9a5f-99b7d1780890 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46ed835b11244870952aaec4b2b6f771'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:48.198807 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1043/2096] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:36:47 2022] DELETE /identity/v3/users/46ed835b11244870952aaec4b2b6f771 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:48.604313 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:48.871738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfe2e38-ae5f-406b-8501-9867a0e37374 service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:48.941710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfe2e38-ae5f-406b-8501-9867a0e37374 service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:49.016020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8743e9e-7511-4279-857d-5f222eb16398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:49.016457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8743e9e-7511-4279-857d-5f222eb16398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:49.028117 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8743e9e-7511-4279-857d-5f222eb16398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:49.030042 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8743e9e-7511-4279-857d-5f222eb16398 admin admin] PATH_INFO: `/v3/projects/6ed9d44c9a094bdc892a908110db882f` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:49.031843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8743e9e-7511-4279-857d-5f222eb16398 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ed9d44c9a094bdc892a908110db882f)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:49.035632 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8743e9e-7511-4279-857d-5f222eb16398 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:49.264152 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-c8743e9e-7511-4279-857d-5f222eb16398 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ed9d44c9a094bdc892a908110db882f', 'request_id': 'req-c8743e9e-7511-4279-857d-5f222eb16398'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:49.398712 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-c8743e9e-7511-4279-857d-5f222eb16398 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ed9d44c9a094bdc892a908110db882f', 'request_id': 'req-c8743e9e-7511-4279-857d-5f222eb16398'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:49.405433 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-c8743e9e-7511-4279-857d-5f222eb16398 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ed9d44c9a094bdc892a908110db882f', 'request_id': 'req-c8743e9e-7511-4279-857d-5f222eb16398'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:49.431144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-c8743e9e-7511-4279-857d-5f222eb16398 admin admin] The token cache is being invalidate because project 6ed9d44c9a094bdc892a908110db882f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:36:49.439984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-c8743e9e-7511-4279-857d-5f222eb16398 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:49.440705 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-c8743e9e-7511-4279-857d-5f222eb16398 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:49.442685 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1054/2097] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:36:48 2022] DELETE /identity/v3/projects/6ed9d44c9a094bdc892a908110db882f => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:49.447802 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd301c-05bf-4d1f-8402-3c1fdf1c57a4 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:49.448507 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd301c-05bf-4d1f-8402-3c1fdf1c57a4 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:49.448507 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd301c-05bf-4d1f-8402-3c1fdf1c57a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:49.466929 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-08fd301c-05bf-4d1f-8402-3c1fdf1c57a4 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:49.468127 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-08fd301c-05bf-4d1f-8402-3c1fdf1c57a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:49.534173 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1044/2098] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:49.542809 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a9baf-5fba-498d-9b85-769e7ec682ab None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:49.543128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a9baf-5fba-498d-9b85-769e7ec682ab None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:49.543610 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a9baf-5fba-498d-9b85-769e7ec682ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:49.571629 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-ab7a9baf-5fba-498d-9b85-769e7ec682ab None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:49.576728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ab7a9baf-5fba-498d-9b85-769e7ec682ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:49.606284 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1055/2099] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:49.607204 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd301c-05bf-4d1f-8402-3c1fdf1c57a4 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:49.618128 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd301c-05bf-4d1f-8402-3c1fdf1c57a4 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:49.624437 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70530dd-f3cb-4727-acf9-9ec4667c94b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:49.625545 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70530dd-f3cb-4727-acf9-9ec4667c94b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:49.626053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70530dd-f3cb-4727-acf9-9ec4667c94b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:49.626490 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70530dd-f3cb-4727-acf9-9ec4667c94b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:49.627954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70530dd-f3cb-4727-acf9-9ec4667c94b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:36:49.631178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70530dd-f3cb-4727-acf9-9ec4667c94b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:49.631178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70530dd-f3cb-4727-acf9-9ec4667c94b5 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:49.645031 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1045/2100] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:36:49.650215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a9baf-5fba-498d-9b85-769e7ec682ab None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:49.660640 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a9baf-5fba-498d-9b85-769e7ec682ab None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:49.668047 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae25c01-07cc-4235-97c7-424906358e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:49.669215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae25c01-07cc-4235-97c7-424906358e15 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:49.669653 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae25c01-07cc-4235-97c7-424906358e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:49.670091 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae25c01-07cc-4235-97c7-424906358e15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:49.671735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae25c01-07cc-4235-97c7-424906358e15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:49.674646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae25c01-07cc-4235-97c7-424906358e15 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:49.829541 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1056/2101] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:36:49 2022] POST /identity/v3/projects => generated 367 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:36:49.835662 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70530dd-f3cb-4727-acf9-9ec4667c94b5 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:49.868395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70530dd-f3cb-4727-acf9-9ec4667c94b5 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:49.971194 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f28d87-7fd6-404c-8b0a-af596a7d0bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:49.972029 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f28d87-7fd6-404c-8b0a-af596a7d0bd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:49.972450 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f28d87-7fd6-404c-8b0a-af596a7d0bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:49.973777 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f28d87-7fd6-404c-8b0a-af596a7d0bd9 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:49.973777 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f28d87-7fd6-404c-8b0a-af596a7d0bd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:49.976311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f28d87-7fd6-404c-8b0a-af596a7d0bd9 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:50.075018 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1046/2102] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:36:49 2022] POST /identity/v3/users => generated 284 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:36:50.077421 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae25c01-07cc-4235-97c7-424906358e15 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:50.095088 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae25c01-07cc-4235-97c7-424906358e15 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:50.121458 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ebba4a-c58e-4238-a7dd-70800bd89a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:50.122993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebba4a-c58e-4238-a7dd-70800bd89a70 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:50.123487 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebba4a-c58e-4238-a7dd-70800bd89a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:50.123981 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebba4a-c58e-4238-a7dd-70800bd89a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:50.125631 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ebba4a-c58e-4238-a7dd-70800bd89a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:50.129186 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ebba4a-c58e-4238-a7dd-70800bd89a70 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:50.145703 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1057/2103] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:50.151941 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f28d87-7fd6-404c-8b0a-af596a7d0bd9 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:50.167394 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f28d87-7fd6-404c-8b0a-af596a7d0bd9 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:50.174326 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276688e4-c933-4038-a3b0-29a7ee67320b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:50.175193 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276688e4-c933-4038-a3b0-29a7ee67320b admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:50.175613 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276688e4-c933-4038-a3b0-29a7ee67320b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:50.176623 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276688e4-c933-4038-a3b0-29a7ee67320b admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:50.177869 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276688e4-c933-4038-a3b0-29a7ee67320b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:50.180019 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276688e4-c933-4038-a3b0-29a7ee67320b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:50.204436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1047/2104] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:50 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:50.208306 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ebba4a-c58e-4238-a7dd-70800bd89a70 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:50.223010 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ebba4a-c58e-4238-a7dd-70800bd89a70 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:50.234720 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b92d62-50c2-4e39-829b-a5b17799a321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:50.235734 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b92d62-50c2-4e39-829b-a5b17799a321 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:50.236212 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b92d62-50c2-4e39-829b-a5b17799a321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:50.239806 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b92d62-50c2-4e39-829b-a5b17799a321 admin admin] PATH_INFO: `/v3/projects/db7f8fadad1549b69dd88e51735eeedb/users/54ec103dc24c408f9761e3bc1f0d4edb/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:50.255290 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b92d62-50c2-4e39-829b-a5b17799a321 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db7f8fadad1549b69dd88e51735eeedb, user_id=54ec103dc24c408f9761e3bc1f0d4edb, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:50.265634 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b92d62-50c2-4e39-829b-a5b17799a321 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:50.362820 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1058/2105] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:36:50 2022] PUT /identity/v3/projects/db7f8fadad1549b69dd88e51735eeedb/users/54ec103dc24c408f9761e3bc1f0d4edb/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:50.373970 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1166999-5383-4188-8def-8c922efe3825 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:50.374561 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1166999-5383-4188-8def-8c922efe3825 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:50.374870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1166999-5383-4188-8def-8c922efe3825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:50.423347 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-f1166999-5383-4188-8def-8c922efe3825 None None] MFA Rules not processed for user `54ec103dc24c408f9761e3bc1f0d4edb`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:50.425282 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f1166999-5383-4188-8def-8c922efe3825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:50.533755 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1048/2106] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:50 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:50.538226 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b92d62-50c2-4e39-829b-a5b17799a321 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:50.647140 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b92d62-50c2-4e39-829b-a5b17799a321 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:50.656167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:50.663599 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c64e74-6ec6-4f4a-a43f-c0003a5c58bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:50.664455 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c64e74-6ec6-4f4a-a43f-c0003a5c58bc admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:50.667473 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c64e74-6ec6-4f4a-a43f-c0003a5c58bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:50.667473 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c64e74-6ec6-4f4a-a43f-c0003a5c58bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:50.667473 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c64e74-6ec6-4f4a-a43f-c0003a5c58bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:50.668203 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c64e74-6ec6-4f4a-a43f-c0003a5c58bc admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:50.804548 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1059/2107] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:36:50 2022] POST /identity/v3/projects => generated 369 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:36:50.808241 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1166999-5383-4188-8def-8c922efe3825 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:50.833314 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1166999-5383-4188-8def-8c922efe3825 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:50.944327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5b39a-52b4-46fe-bac4-b219f6e3e34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:50.954929 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5b39a-52b4-46fe-bac4-b219f6e3e34b admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:50.954929 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5b39a-52b4-46fe-bac4-b219f6e3e34b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:50.954929 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5b39a-52b4-46fe-bac4-b219f6e3e34b admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:50.954929 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d5b39a-52b4-46fe-bac4-b219f6e3e34b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:50.954929 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d5b39a-52b4-46fe-bac4-b219f6e3e34b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:51.051344 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c64e74-6ec6-4f4a-a43f-c0003a5c58bc admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:51.057316 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1049/2108] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:36:50 2022] POST /identity/v3/users => generated 291 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:36:51.063473 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c64e74-6ec6-4f4a-a43f-c0003a5c58bc admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:51.070456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b482d-d76e-4836-930c-c92fa5f139e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:51.071381 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b482d-d76e-4836-930c-c92fa5f139e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:51.078727 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b482d-d76e-4836-930c-c92fa5f139e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:51.078727 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b482d-d76e-4836-930c-c92fa5f139e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:51.078727 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b482d-d76e-4836-930c-c92fa5f139e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:51.078727 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b482d-d76e-4836-930c-c92fa5f139e1 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:51.092044 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5b39a-52b4-46fe-bac4-b219f6e3e34b admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:51.092458 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1060/2109] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:51.109792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5b39a-52b4-46fe-bac4-b219f6e3e34b admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:51.114387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddf22d-81b5-4a1a-a610-cd62e5321bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:51.115416 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddf22d-81b5-4a1a-a610-cd62e5321bb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:51.115639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddf22d-81b5-4a1a-a610-cd62e5321bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:51.116064 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddf22d-81b5-4a1a-a610-cd62e5321bb8 admin admin] PATH_INFO: `/v3/projects/2fb15fc9a31f41e0929deadc4d6e7829/users/9ee3e982309f4227a003703aebde507d/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:51.132118 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ddf22d-81b5-4a1a-a610-cd62e5321bb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fb15fc9a31f41e0929deadc4d6e7829, user_id=9ee3e982309f4227a003703aebde507d, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:51.135553 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ddf22d-81b5-4a1a-a610-cd62e5321bb8 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:51.228870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1050/2110] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:36:51 2022] PUT /identity/v3/projects/2fb15fc9a31f41e0929deadc4d6e7829/users/9ee3e982309f4227a003703aebde507d/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:51.232978 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b482d-d76e-4836-930c-c92fa5f139e1 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:51.254994 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b482d-d76e-4836-930c-c92fa5f139e1 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:51.343040 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934c8d8-f360-47f5-95fa-ac5f7540250f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:51.343984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934c8d8-f360-47f5-95fa-ac5f7540250f admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:51.345249 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934c8d8-f360-47f5-95fa-ac5f7540250f admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:51.345682 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934c8d8-f360-47f5-95fa-ac5f7540250f admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:51.346641 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934c8d8-f360-47f5-95fa-ac5f7540250f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:51.348545 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934c8d8-f360-47f5-95fa-ac5f7540250f admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:51.369523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1061/2111] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:51.371554 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddf22d-81b5-4a1a-a610-cd62e5321bb8 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:51.386543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddf22d-81b5-4a1a-a610-cd62e5321bb8 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:51.392636 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8e142-05be-426c-b62b-65d863d2d766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:51.393724 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8e142-05be-426c-b62b-65d863d2d766 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:51.394302 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8e142-05be-426c-b62b-65d863d2d766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:51.394722 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8e142-05be-426c-b62b-65d863d2d766 admin admin] PATH_INFO: `/v3/projects/2fb15fc9a31f41e0929deadc4d6e7829/users/9ee3e982309f4227a003703aebde507d/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:51.397769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf8e142-05be-426c-b62b-65d863d2d766 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fb15fc9a31f41e0929deadc4d6e7829, user_id=9ee3e982309f4227a003703aebde507d, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:51.401374 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf8e142-05be-426c-b62b-65d863d2d766 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:51.427455 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1051/2112] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:36:51 2022] PUT /identity/v3/projects/2fb15fc9a31f41e0929deadc4d6e7829/users/9ee3e982309f4227a003703aebde507d/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:51.429291 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6e8c5-5a41-45bf-939c-2cd26bbb8520 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:51.429742 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6e8c5-5a41-45bf-939c-2cd26bbb8520 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:51.430219 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6e8c5-5a41-45bf-939c-2cd26bbb8520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:51.466254 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-1ed6e8c5-5a41-45bf-939c-2cd26bbb8520 None None] MFA Rules not processed for user `9ee3e982309f4227a003703aebde507d`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:51.467954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1ed6e8c5-5a41-45bf-939c-2cd26bbb8520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:51.576190 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:51.593138 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1062/2113] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:51 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:51.607642 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8e142-05be-426c-b62b-65d863d2d766 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:51.609487 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-8cf8e142-05be-426c-b62b-65d863d2d766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:51.757639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8e142-05be-426c-b62b-65d863d2d766 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:51.766291 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62334e-fd99-48c5-a286-e3273a1404bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:51.766999 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62334e-fd99-48c5-a286-e3273a1404bd service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:51.767460 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62334e-fd99-48c5-a286-e3273a1404bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:51.767855 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62334e-fd99-48c5-a286-e3273a1404bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:51.786363 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62334e-fd99-48c5-a286-e3273a1404bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:51.788673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62334e-fd99-48c5-a286-e3273a1404bd service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:51.805565 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1052/2114] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:36:51 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:52.042675 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6e8c5-5a41-45bf-939c-2cd26bbb8520 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:52.044785 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1ed6e8c5-5a41-45bf-939c-2cd26bbb8520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:52.091748 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6e8c5-5a41-45bf-939c-2cd26bbb8520 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:52.109449 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008726f0-dee0-4916-b3cb-8e7e17f673fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:52.110389 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008726f0-dee0-4916-b3cb-8e7e17f673fe service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:52.112161 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008726f0-dee0-4916-b3cb-8e7e17f673fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:52.112702 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008726f0-dee0-4916-b3cb-8e7e17f673fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:52.251571 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:52.369213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008726f0-dee0-4916-b3cb-8e7e17f673fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:52.380371 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008726f0-dee0-4916-b3cb-8e7e17f673fe service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:52.390534 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1063/2115] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:36:52 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:53.461376 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:54.099433 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc1554-29c1-4b01-aaf1-c26adeef9d2f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:54.100587 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc1554-29c1-4b01-aaf1-c26adeef9d2f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:54.100868 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc1554-29c1-4b01-aaf1-c26adeef9d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:54.143590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-06cc1554-29c1-4b01-aaf1-c26adeef9d2f None None] MFA Rules not processed for user `9001a4b10d544226b3b1453b06883db4`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:54.152719 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-06cc1554-29c1-4b01-aaf1-c26adeef9d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:54.321431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1053/2116] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:54.325907 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008726f0-dee0-4916-b3cb-8e7e17f673fe service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:54.328408 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-008726f0-dee0-4916-b3cb-8e7e17f673fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:54.517391 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008726f0-dee0-4916-b3cb-8e7e17f673fe service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:54.523475 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c05dd-4694-4905-a514-1e1e1a8e7600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:54.524261 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c05dd-4694-4905-a514-1e1e1a8e7600 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:54.524642 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c05dd-4694-4905-a514-1e1e1a8e7600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:54.525097 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c05dd-4694-4905-a514-1e1e1a8e7600 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:54.526101 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c05dd-4694-4905-a514-1e1e1a8e7600 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:54.529414 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c05dd-4694-4905-a514-1e1e1a8e7600 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:54.679604 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:54.695987 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1064/2117] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:36:54 2022] POST /identity/v3/projects => generated 371 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:36:54.699901 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc1554-29c1-4b01-aaf1-c26adeef9d2f None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:54.727295 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc1554-29c1-4b01-aaf1-c26adeef9d2f None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:54.814365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:54.825467 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ccc5d-87cc-4299-801c-3a3c476fe322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:54.826153 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ccc5d-87cc-4299-801c-3a3c476fe322 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:54.826456 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ccc5d-87cc-4299-801c-3a3c476fe322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:54.827029 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ccc5d-87cc-4299-801c-3a3c476fe322 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:54.827724 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ccc5d-87cc-4299-801c-3a3c476fe322 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:54.829724 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ccc5d-87cc-4299-801c-3a3c476fe322 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:54.932873 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1054/2118] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:36:54 2022] POST /identity/v3/users => generated 292 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:36:54.935640 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c05dd-4694-4905-a514-1e1e1a8e7600 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:54.970403 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c05dd-4694-4905-a514-1e1e1a8e7600 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:54.991161 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df4096-fca7-425d-a0ee-560b46150c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:54.991161 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df4096-fca7-425d-a0ee-560b46150c4c admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:54.991161 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df4096-fca7-425d-a0ee-560b46150c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:54.991161 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df4096-fca7-425d-a0ee-560b46150c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:54.994506 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df4096-fca7-425d-a0ee-560b46150c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:54.997656 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df4096-fca7-425d-a0ee-560b46150c4c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:55.039645 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1065/2119] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:55.044354 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ccc5d-87cc-4299-801c-3a3c476fe322 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:55.058029 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ccc5d-87cc-4299-801c-3a3c476fe322 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:55.080906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f03a1-2161-4845-ab18-b35522a83354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:55.085349 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f03a1-2161-4845-ab18-b35522a83354 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:55.085349 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f03a1-2161-4845-ab18-b35522a83354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:55.085349 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4f03a1-2161-4845-ab18-b35522a83354 admin admin] PATH_INFO: `/v3/projects/605d09ab997143aabb55d2733c74116f/users/3dae32fa077840d9a053b6bef162615e/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:55.114129 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4f03a1-2161-4845-ab18-b35522a83354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=605d09ab997143aabb55d2733c74116f, user_id=3dae32fa077840d9a053b6bef162615e, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:55.118786 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4f03a1-2161-4845-ab18-b35522a83354 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:55.223037 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1055/2120] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:36:55 2022] PUT /identity/v3/projects/605d09ab997143aabb55d2733c74116f/users/3dae32fa077840d9a053b6bef162615e/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:55.227480 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df4096-fca7-425d-a0ee-560b46150c4c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:55.240075 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df4096-fca7-425d-a0ee-560b46150c4c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:55.340127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5aac0-cc8a-4776-bd4a-f933e0951a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:55.341046 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5aac0-cc8a-4776-bd4a-f933e0951a93 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:55.341559 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5aac0-cc8a-4776-bd4a-f933e0951a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:55.341975 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5aac0-cc8a-4776-bd4a-f933e0951a93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:55.342807 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec5aac0-cc8a-4776-bd4a-f933e0951a93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:55.344475 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec5aac0-cc8a-4776-bd4a-f933e0951a93 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:55.355817 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1066/2121] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:36:55.360133 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f03a1-2161-4845-ab18-b35522a83354 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:55.371360 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4f03a1-2161-4845-ab18-b35522a83354 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:55.376019 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253ad51-90c2-4f1f-81c7-d4c15c876293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:55.376908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253ad51-90c2-4f1f-81c7-d4c15c876293 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:55.377191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253ad51-90c2-4f1f-81c7-d4c15c876293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:55.377469 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253ad51-90c2-4f1f-81c7-d4c15c876293 admin admin] PATH_INFO: `/v3/projects/605d09ab997143aabb55d2733c74116f/users/3dae32fa077840d9a053b6bef162615e/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:55.379552 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1253ad51-90c2-4f1f-81c7-d4c15c876293 admin admin] RBAC: Authorizing `identity:create_grant(project_id=605d09ab997143aabb55d2733c74116f, user_id=3dae32fa077840d9a053b6bef162615e, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:55.382195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1253ad51-90c2-4f1f-81c7-d4c15c876293 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:55.396459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1056/2122] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:36:55 2022] PUT /identity/v3/projects/605d09ab997143aabb55d2733c74116f/users/3dae32fa077840d9a053b6bef162615e/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:56.482922 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:57.450503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:58.615783 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:58.971869 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5aac0-cc8a-4776-bd4a-f933e0951a93 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:58.982166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5aac0-cc8a-4776-bd4a-f933e0951a93 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:59.045376 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef5fb5-dd3b-43be-b72f-c2c04741bf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:59.046529 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef5fb5-dd3b-43be-b72f-c2c04741bf9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:59.047032 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef5fb5-dd3b-43be-b72f-c2c04741bf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:59.047436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef5fb5-dd3b-43be-b72f-c2c04741bf9a admin admin] PATH_INFO: `/v3/users/54ec103dc24c408f9761e3bc1f0d4edb` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:59.049221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ef5fb5-dd3b-43be-b72f-c2c04741bf9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=54ec103dc24c408f9761e3bc1f0d4edb)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:59.051430 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ef5fb5-dd3b-43be-b72f-c2c04741bf9a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:59.331133 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-78ef5fb5-dd3b-43be-b72f-c2c04741bf9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54ec103dc24c408f9761e3bc1f0d4edb'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:59.444438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-78ef5fb5-dd3b-43be-b72f-c2c04741bf9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54ec103dc24c408f9761e3bc1f0d4edb'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:59.470996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-78ef5fb5-dd3b-43be-b72f-c2c04741bf9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54ec103dc24c408f9761e3bc1f0d4edb'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:59.471805 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-78ef5fb5-dd3b-43be-b72f-c2c04741bf9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54ec103dc24c408f9761e3bc1f0d4edb'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:59.485418 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-78ef5fb5-dd3b-43be-b72f-c2c04741bf9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54ec103dc24c408f9761e3bc1f0d4edb'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:59.494859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1067/2123] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:36:58 2022] DELETE /identity/v3/users/54ec103dc24c408f9761e3bc1f0d4edb => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:59.504490 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17972d-ab0e-4b3c-890f-eeef29f0bf4b None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:59.504690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17972d-ab0e-4b3c-890f-eeef29f0bf4b None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:59.505161 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc17972d-ab0e-4b3c-890f-eeef29f0bf4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:59.557137 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-cc17972d-ab0e-4b3c-890f-eeef29f0bf4b None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:36:59.559827 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-cc17972d-ab0e-4b3c-890f-eeef29f0bf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:59.654720 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1057/2124] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:36:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:36:59.676694 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef5fb5-dd3b-43be-b72f-c2c04741bf9a admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:59.679097 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-78ef5fb5-dd3b-43be-b72f-c2c04741bf9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:59.832142 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef5fb5-dd3b-43be-b72f-c2c04741bf9a admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:59.838257 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b27219-eac6-457a-b11c-4e75264d2b36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:59.839330 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b27219-eac6-457a-b11c-4e75264d2b36 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:59.840424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b27219-eac6-457a-b11c-4e75264d2b36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:59.841197 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b27219-eac6-457a-b11c-4e75264d2b36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:59.857113 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b27219-eac6-457a-b11c-4e75264d2b36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:59.859184 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b27219-eac6-457a-b11c-4e75264d2b36 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:59.874425 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1068/2125] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:36:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:00.273171 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:00.658822 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc17972d-ab0e-4b3c-890f-eeef29f0bf4b None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:00.660302 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-cc17972d-ab0e-4b3c-890f-eeef29f0bf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:00.673335 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc17972d-ab0e-4b3c-890f-eeef29f0bf4b None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:00.678668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d790d89-3e84-4245-b705-bae986710699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:00.679518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d790d89-3e84-4245-b705-bae986710699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:00.679795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d790d89-3e84-4245-b705-bae986710699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:00.680242 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d790d89-3e84-4245-b705-bae986710699 admin admin] PATH_INFO: `/v3/users/1b3ff9487a154831afa6ae492a0b0f7c` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:00.681535 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d790d89-3e84-4245-b705-bae986710699 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b3ff9487a154831afa6ae492a0b0f7c)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:00.683460 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d790d89-3e84-4245-b705-bae986710699 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:00.719257 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:00.935214 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-3d790d89-3e84-4245-b705-bae986710699 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b3ff9487a154831afa6ae492a0b0f7c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:00.952892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b27219-eac6-457a-b11c-4e75264d2b36 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:00.954181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d2b27219-eac6-457a-b11c-4e75264d2b36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:00.967820 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b27219-eac6-457a-b11c-4e75264d2b36 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:00.973281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e27b70-f292-4bbd-85d5-856aac01efc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:00.974090 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e27b70-f292-4bbd-85d5-856aac01efc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:00.974375 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e27b70-f292-4bbd-85d5-856aac01efc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:00.974725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e27b70-f292-4bbd-85d5-856aac01efc3 admin admin] PATH_INFO: `/v3/projects/db7f8fadad1549b69dd88e51735eeedb` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:00.977154 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e27b70-f292-4bbd-85d5-856aac01efc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db7f8fadad1549b69dd88e51735eeedb)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:00.990953 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e27b70-f292-4bbd-85d5-856aac01efc3 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:01.032385 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-3d790d89-3e84-4245-b705-bae986710699 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b3ff9487a154831afa6ae492a0b0f7c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:01.033058 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-3d790d89-3e84-4245-b705-bae986710699 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b3ff9487a154831afa6ae492a0b0f7c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:01.043580 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-3d790d89-3e84-4245-b705-bae986710699 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b3ff9487a154831afa6ae492a0b0f7c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:01.046199 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-3d790d89-3e84-4245-b705-bae986710699 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b3ff9487a154831afa6ae492a0b0f7c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:01.082322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1058/2126] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:37:00 2022] DELETE /identity/v3/users/1b3ff9487a154831afa6ae492a0b0f7c => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:01.084712 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91d8bc-c341-41c8-a9e4-b5fb81f0ec3f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:01.085253 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91d8bc-c341-41c8-a9e4-b5fb81f0ec3f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:01.085675 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91d8bc-c341-41c8-a9e4-b5fb81f0ec3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:01.122240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-ee91d8bc-c341-41c8-a9e4-b5fb81f0ec3f None None] MFA Rules not processed for user `dac1130950b8457ea3a9349346437357`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:01.124042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ee91d8bc-c341-41c8-a9e4-b5fb81f0ec3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:01.124931 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-37e27b70-f292-4bbd-85d5-856aac01efc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db7f8fadad1549b69dd88e51735eeedb', 'request_id': 'req-37e27b70-f292-4bbd-85d5-856aac01efc3'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:01.203886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1059/2127] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:01.205355 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91d8bc-c341-41c8-a9e4-b5fb81f0ec3f None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:01.207323 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ee91d8bc-c341-41c8-a9e4-b5fb81f0ec3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:01.227445 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-37e27b70-f292-4bbd-85d5-856aac01efc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db7f8fadad1549b69dd88e51735eeedb', 'request_id': 'req-37e27b70-f292-4bbd-85d5-856aac01efc3'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:01.235240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-37e27b70-f292-4bbd-85d5-856aac01efc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db7f8fadad1549b69dd88e51735eeedb', 'request_id': 'req-37e27b70-f292-4bbd-85d5-856aac01efc3'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:01.262991 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-37e27b70-f292-4bbd-85d5-856aac01efc3 admin admin] The token cache is being invalidate because project db7f8fadad1549b69dd88e51735eeedb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:37:01.262991 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-37e27b70-f292-4bbd-85d5-856aac01efc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:01.263898 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-37e27b70-f292-4bbd-85d5-856aac01efc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:01.269805 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1069/2128] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:37:00 2022] DELETE /identity/v3/projects/db7f8fadad1549b69dd88e51735eeedb => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:37:01.271499 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f6ecd-7b19-4713-a52a-729ec2394448 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:01.272520 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f6ecd-7b19-4713-a52a-729ec2394448 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:01.273020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f6ecd-7b19-4713-a52a-729ec2394448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:01.360037 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-1e1f6ecd-7b19-4713-a52a-729ec2394448 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:01.361263 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1e1f6ecd-7b19-4713-a52a-729ec2394448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:01.368611 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91d8bc-c341-41c8-a9e4-b5fb81f0ec3f None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:01.381391 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660f4ce-6487-4f0b-aca6-766e41195c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:01.382867 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660f4ce-6487-4f0b-aca6-766e41195c9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:01.382867 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660f4ce-6487-4f0b-aca6-766e41195c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:01.386357 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660f4ce-6487-4f0b-aca6-766e41195c9b admin admin] PATH_INFO: `/v3/users/ba54577bec664442a1bd8d4ac3b9085b` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:01.388519 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9660f4ce-6487-4f0b-aca6-766e41195c9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba54577bec664442a1bd8d4ac3b9085b)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:01.392374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1070/2129] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:01.393225 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f6ecd-7b19-4713-a52a-729ec2394448 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:01.397234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9660f4ce-6487-4f0b-aca6-766e41195c9b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:01.399314 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1e1f6ecd-7b19-4713-a52a-729ec2394448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:01.435588 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f6ecd-7b19-4713-a52a-729ec2394448 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:01.452651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8b123-656c-4f7b-8f25-ed694b0a2848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:01.453539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8b123-656c-4f7b-8f25-ed694b0a2848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:01.453817 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8b123-656c-4f7b-8f25-ed694b0a2848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:01.454268 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8b123-656c-4f7b-8f25-ed694b0a2848 admin admin] PATH_INFO: `/v3/users/9ee3e982309f4227a003703aebde507d` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:01.460140 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a8b123-656c-4f7b-8f25-ed694b0a2848 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ee3e982309f4227a003703aebde507d)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:01.462116 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a8b123-656c-4f7b-8f25-ed694b0a2848 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:01.676423 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-9660f4ce-6487-4f0b-aca6-766e41195c9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba54577bec664442a1bd8d4ac3b9085b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:01.705886 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-70a8b123-656c-4f7b-8f25-ed694b0a2848 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ee3e982309f4227a003703aebde507d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:01.832812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-9660f4ce-6487-4f0b-aca6-766e41195c9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba54577bec664442a1bd8d4ac3b9085b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:01.833631 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-9660f4ce-6487-4f0b-aca6-766e41195c9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba54577bec664442a1bd8d4ac3b9085b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:01.844777 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-9660f4ce-6487-4f0b-aca6-766e41195c9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba54577bec664442a1bd8d4ac3b9085b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:01.845761 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-9660f4ce-6487-4f0b-aca6-766e41195c9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba54577bec664442a1bd8d4ac3b9085b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:01.868135 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1060/2130] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:37:01 2022] DELETE /identity/v3/users/ba54577bec664442a1bd8d4ac3b9085b => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:01.869989 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660f4ce-6487-4f0b-aca6-766e41195c9b admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:01.872470 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-9660f4ce-6487-4f0b-aca6-766e41195c9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:01.938549 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660f4ce-6487-4f0b-aca6-766e41195c9b admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:01.942496 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983c62f-6b3f-4df2-af71-d8e2883c1bf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:01.943185 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983c62f-6b3f-4df2-af71-d8e2883c1bf1 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:01.943496 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983c62f-6b3f-4df2-af71-d8e2883c1bf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:01.943772 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3983c62f-6b3f-4df2-af71-d8e2883c1bf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:01.945597 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3983c62f-6b3f-4df2-af71-d8e2883c1bf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:01.957301 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-70a8b123-656c-4f7b-8f25-ed694b0a2848 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ee3e982309f4227a003703aebde507d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:01.973286 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-70a8b123-656c-4f7b-8f25-ed694b0a2848 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ee3e982309f4227a003703aebde507d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:01.974326 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-70a8b123-656c-4f7b-8f25-ed694b0a2848 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ee3e982309f4227a003703aebde507d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:01.982734 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-70a8b123-656c-4f7b-8f25-ed694b0a2848 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ee3e982309f4227a003703aebde507d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:01.990118 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1071/2131] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:37:01 2022] DELETE /identity/v3/users/9ee3e982309f4227a003703aebde507d => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:01.992719 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ce4e1-adb4-435b-a37e-d9f4bf50b58a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:01.993261 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ce4e1-adb4-435b-a37e-d9f4bf50b58a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:01.993850 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ce4e1-adb4-435b-a37e-d9f4bf50b58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:02.014605 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-881ce4e1-adb4-435b-a37e-d9f4bf50b58a None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:02.016375 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-881ce4e1-adb4-435b-a37e-d9f4bf50b58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:02.041659 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3983c62f-6b3f-4df2-af71-d8e2883c1bf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:02.045539 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3983c62f-6b3f-4df2-af71-d8e2883c1bf1 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:02.119234 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1072/2132] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:02.126615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1061/2133] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:02.133603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ce4e1-adb4-435b-a37e-d9f4bf50b58a None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:02.142540 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-881ce4e1-adb4-435b-a37e-d9f4bf50b58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:02.238506 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ce4e1-adb4-435b-a37e-d9f4bf50b58a None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:02.243654 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e27c19-711a-4797-bd8b-59d4150a08c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:02.244312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e27c19-711a-4797-bd8b-59d4150a08c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:02.244596 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e27c19-711a-4797-bd8b-59d4150a08c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:02.245025 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e27c19-711a-4797-bd8b-59d4150a08c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:02.256925 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e27c19-711a-4797-bd8b-59d4150a08c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:02.258514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e27c19-711a-4797-bd8b-59d4150a08c6 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:02.295375 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1073/2134] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:02.704690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983c62f-6b3f-4df2-af71-d8e2883c1bf1 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:02.706483 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3983c62f-6b3f-4df2-af71-d8e2883c1bf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:02.722676 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3983c62f-6b3f-4df2-af71-d8e2883c1bf1 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:02.738815 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea19ff7-fae8-4d34-91e0-50b6bcde11bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:02.739941 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea19ff7-fae8-4d34-91e0-50b6bcde11bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:02.740382 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea19ff7-fae8-4d34-91e0-50b6bcde11bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:02.741300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea19ff7-fae8-4d34-91e0-50b6bcde11bb admin admin] PATH_INFO: `/v3/projects/2fb15fc9a31f41e0929deadc4d6e7829` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:02.742295 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea19ff7-fae8-4d34-91e0-50b6bcde11bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fb15fc9a31f41e0929deadc4d6e7829)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:02.744687 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea19ff7-fae8-4d34-91e0-50b6bcde11bb admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:02.945501 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-8ea19ff7-fae8-4d34-91e0-50b6bcde11bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fb15fc9a31f41e0929deadc4d6e7829', 'request_id': 'req-8ea19ff7-fae8-4d34-91e0-50b6bcde11bb'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:03.036231 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-8ea19ff7-fae8-4d34-91e0-50b6bcde11bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fb15fc9a31f41e0929deadc4d6e7829', 'request_id': 'req-8ea19ff7-fae8-4d34-91e0-50b6bcde11bb'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:03.045625 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-8ea19ff7-fae8-4d34-91e0-50b6bcde11bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fb15fc9a31f41e0929deadc4d6e7829', 'request_id': 'req-8ea19ff7-fae8-4d34-91e0-50b6bcde11bb'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:03.081246 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-8ea19ff7-fae8-4d34-91e0-50b6bcde11bb admin admin] The token cache is being invalidate because project 2fb15fc9a31f41e0929deadc4d6e7829 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:37:03.081544 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-8ea19ff7-fae8-4d34-91e0-50b6bcde11bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:03.082223 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-8ea19ff7-fae8-4d34-91e0-50b6bcde11bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:03.087267 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1062/2135] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:37:02 2022] DELETE /identity/v3/projects/2fb15fc9a31f41e0929deadc4d6e7829 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:03.089217 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54833073-b9c0-4c3d-98a6-9931602060d8 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:03.089653 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54833073-b9c0-4c3d-98a6-9931602060d8 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:03.089840 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54833073-b9c0-4c3d-98a6-9931602060d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:03.161037 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-54833073-b9c0-4c3d-98a6-9931602060d8 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:03.161037 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-54833073-b9c0-4c3d-98a6-9931602060d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:03.294477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1074/2136] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:03.301214 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a1a70-787f-4439-9dcb-9710a76cc48b None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:03.301568 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a1a70-787f-4439-9dcb-9710a76cc48b None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:03.305474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a1a70-787f-4439-9dcb-9710a76cc48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:03.340130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54833073-b9c0-4c3d-98a6-9931602060d8 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:03.346054 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-54833073-b9c0-4c3d-98a6-9931602060d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:03.391032 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54833073-b9c0-4c3d-98a6-9931602060d8 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:03.407139 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0cfe26-d62b-4617-ba18-6a4c073a847d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:03.408264 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0cfe26-d62b-4617-ba18-6a4c073a847d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:03.409183 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0cfe26-d62b-4617-ba18-6a4c073a847d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:03.409624 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0cfe26-d62b-4617-ba18-6a4c073a847d admin admin] PATH_INFO: `/v3/projects/ed4021fb1d7e465d8439f72923f5d651` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:03.412706 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0cfe26-d62b-4617-ba18-6a4c073a847d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed4021fb1d7e465d8439f72923f5d651)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:03.415373 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0cfe26-d62b-4617-ba18-6a4c073a847d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:03.427102 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-319a1a70-787f-4439-9dcb-9710a76cc48b None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:03.428403 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-319a1a70-787f-4439-9dcb-9710a76cc48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:03.478603 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1063/2137] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:03.480768 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a1a70-787f-4439-9dcb-9710a76cc48b None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:03.496573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a1a70-787f-4439-9dcb-9710a76cc48b None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:03.501808 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544eee4-7375-41a0-87e4-045938154164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:03.502739 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544eee4-7375-41a0-87e4-045938154164 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:03.503279 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544eee4-7375-41a0-87e4-045938154164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:03.503766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544eee4-7375-41a0-87e4-045938154164 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:03.504712 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4544eee4-7375-41a0-87e4-045938154164 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:37:03.505700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4544eee4-7375-41a0-87e4-045938154164 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:03.512580 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4544eee4-7375-41a0-87e4-045938154164 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:03.532465 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1064/2138] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:37:03.536476 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544eee4-7375-41a0-87e4-045938154164 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:03.549663 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-4544eee4-7375-41a0-87e4-045938154164 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:03.572432 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544eee4-7375-41a0-87e4-045938154164 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:03.581998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ed6a2-f447-417b-8b78-9482d754b56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:03.583129 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ed6a2-f447-417b-8b78-9482d754b56b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:03.583482 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ed6a2-f447-417b-8b78-9482d754b56b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:03.583822 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801ed6a2-f447-417b-8b78-9482d754b56b admin admin] PATH_INFO: `/v3/projects/d069fd1d39354fc6a02b0f76de7dffe7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:03.588294 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ed6a2-f447-417b-8b78-9482d754b56b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d069fd1d39354fc6a02b0f76de7dffe7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:03.593718 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801ed6a2-f447-417b-8b78-9482d754b56b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:03.650646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ef0cfe26-d62b-4617-ba18-6a4c073a847d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed4021fb1d7e465d8439f72923f5d651', 'request_id': 'req-ef0cfe26-d62b-4617-ba18-6a4c073a847d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:03.765278 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ef0cfe26-d62b-4617-ba18-6a4c073a847d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed4021fb1d7e465d8439f72923f5d651', 'request_id': 'req-ef0cfe26-d62b-4617-ba18-6a4c073a847d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:03.775852 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ef0cfe26-d62b-4617-ba18-6a4c073a847d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed4021fb1d7e465d8439f72923f5d651', 'request_id': 'req-ef0cfe26-d62b-4617-ba18-6a4c073a847d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:03.794169 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ef0cfe26-d62b-4617-ba18-6a4c073a847d admin admin] The token cache is being invalidate because project ed4021fb1d7e465d8439f72923f5d651 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:37:03.795041 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ef0cfe26-d62b-4617-ba18-6a4c073a847d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:03.796438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ef0cfe26-d62b-4617-ba18-6a4c073a847d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:03.798629 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1075/2139] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:37:03 2022] DELETE /identity/v3/projects/ed4021fb1d7e465d8439f72923f5d651 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:03.800892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0cfe26-d62b-4617-ba18-6a4c073a847d admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:03.804160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ef0cfe26-d62b-4617-ba18-6a4c073a847d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:03.804291 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-801ed6a2-f447-417b-8b78-9482d754b56b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd069fd1d39354fc6a02b0f76de7dffe7', 'request_id': 'req-801ed6a2-f447-417b-8b78-9482d754b56b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:03.915510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0cfe26-d62b-4617-ba18-6a4c073a847d admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:03.918431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-801ed6a2-f447-417b-8b78-9482d754b56b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd069fd1d39354fc6a02b0f76de7dffe7', 'request_id': 'req-801ed6a2-f447-417b-8b78-9482d754b56b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:03.930872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-801ed6a2-f447-417b-8b78-9482d754b56b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd069fd1d39354fc6a02b0f76de7dffe7', 'request_id': 'req-801ed6a2-f447-417b-8b78-9482d754b56b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:03.965860 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-801ed6a2-f447-417b-8b78-9482d754b56b admin admin] The token cache is being invalidate because project d069fd1d39354fc6a02b0f76de7dffe7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:37:03.965860 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-801ed6a2-f447-417b-8b78-9482d754b56b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:03.965860 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-801ed6a2-f447-417b-8b78-9482d754b56b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:03.965860 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1065/2140] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:37:03 2022] DELETE /identity/v3/projects/d069fd1d39354fc6a02b0f76de7dffe7 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:03.965860 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ed6a2-f447-417b-8b78-9482d754b56b admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:03.966260 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-801ed6a2-f447-417b-8b78-9482d754b56b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:04.046717 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e10952-3d9f-42fa-8c10-be9476da71ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:04.047704 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e10952-3d9f-42fa-8c10-be9476da71ee admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:04.048042 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e10952-3d9f-42fa-8c10-be9476da71ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:04.048427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e10952-3d9f-42fa-8c10-be9476da71ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:04.049629 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e10952-3d9f-42fa-8c10-be9476da71ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:04.051954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e10952-3d9f-42fa-8c10-be9476da71ee admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:04.070963 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801ed6a2-f447-417b-8b78-9482d754b56b admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:04.080677 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb71fb1-41fa-4e3e-b731-788a17b987bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:04.091351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb71fb1-41fa-4e3e-b731-788a17b987bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:04.099130 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb71fb1-41fa-4e3e-b731-788a17b987bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:04.099130 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb71fb1-41fa-4e3e-b731-788a17b987bc admin admin] PATH_INFO: `/v3/users/d529f0143eb74946a719e301cab5fe43` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:04.100213 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb71fb1-41fa-4e3e-b731-788a17b987bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d529f0143eb74946a719e301cab5fe43)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:04.104673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb71fb1-41fa-4e3e-b731-788a17b987bc admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:04.237294 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1076/2141] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:37:03 2022] POST /identity/v3/projects => generated 381 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:04.238938 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbdc5b-029d-43db-8da5-20198921a5f7 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:04.239304 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbdc5b-029d-43db-8da5-20198921a5f7 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:04.239660 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cbdc5b-029d-43db-8da5-20198921a5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:04.283557 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-c8cbdc5b-029d-43db-8da5-20198921a5f7 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:04.283557 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c8cbdc5b-029d-43db-8da5-20198921a5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:04.390139 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1077/2142] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:04.396671 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cbdc5b-029d-43db-8da5-20198921a5f7 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:04.396671 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c8cbdc5b-029d-43db-8da5-20198921a5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:04.420468 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cbdc5b-029d-43db-8da5-20198921a5f7 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:04.430117 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f35b8-5829-4a9d-a1bc-5a218d7d44dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:04.430855 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f35b8-5829-4a9d-a1bc-5a218d7d44dc admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:04.431148 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f35b8-5829-4a9d-a1bc-5a218d7d44dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:04.431597 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f35b8-5829-4a9d-a1bc-5a218d7d44dc admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:04.432311 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f35b8-5829-4a9d-a1bc-5a218d7d44dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:04.443917 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f35b8-5829-4a9d-a1bc-5a218d7d44dc admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:04.444160 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-deb71fb1-41fa-4e3e-b731-788a17b987bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd529f0143eb74946a719e301cab5fe43'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:04.604429 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-deb71fb1-41fa-4e3e-b731-788a17b987bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd529f0143eb74946a719e301cab5fe43'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:04.605366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-deb71fb1-41fa-4e3e-b731-788a17b987bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd529f0143eb74946a719e301cab5fe43'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:04.617082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1078/2143] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:37:04 2022] POST /identity/v3/users => generated 291 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:04.621839 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c762157-45a0-4d99-b18a-a06becd6dd94 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:04.623043 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c762157-45a0-4d99-b18a-a06becd6dd94 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:04.623368 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c762157-45a0-4d99-b18a-a06becd6dd94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:04.632137 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-deb71fb1-41fa-4e3e-b731-788a17b987bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd529f0143eb74946a719e301cab5fe43'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:04.637124 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-deb71fb1-41fa-4e3e-b731-788a17b987bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd529f0143eb74946a719e301cab5fe43'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:04.684153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-6c762157-45a0-4d99-b18a-a06becd6dd94 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:04.685525 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6c762157-45a0-4d99-b18a-a06becd6dd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:04.707208 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1079/2144] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:04.708694 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c762157-45a0-4d99-b18a-a06becd6dd94 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:04.711154 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6c762157-45a0-4d99-b18a-a06becd6dd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:04.712255 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1066/2145] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:37:03 2022] DELETE /identity/v3/users/d529f0143eb74946a719e301cab5fe43 => generated 0 bytes in 751 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:04.713234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb71fb1-41fa-4e3e-b731-788a17b987bc admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:04.737766 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c762157-45a0-4d99-b18a-a06becd6dd94 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:04.821886 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8604744-ee87-4938-a78d-7dce339d89d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:04.822963 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8604744-ee87-4938-a78d-7dce339d89d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:04.823230 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8604744-ee87-4938-a78d-7dce339d89d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:04.823593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8604744-ee87-4938-a78d-7dce339d89d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:04.825178 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8604744-ee87-4938-a78d-7dce339d89d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:04.827476 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8604744-ee87-4938-a78d-7dce339d89d9 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:04.845017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1080/2146] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:04.849554 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8604744-ee87-4938-a78d-7dce339d89d9 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:04.873561 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb71fb1-41fa-4e3e-b731-788a17b987bc admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:04.886466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df264c-bd02-476d-bc98-1a30ec2a4d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:04.887194 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df264c-bd02-476d-bc98-1a30ec2a4d7e admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:04.887482 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df264c-bd02-476d-bc98-1a30ec2a4d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:04.887758 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df264c-bd02-476d-bc98-1a30ec2a4d7e admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:04.888501 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04df264c-bd02-476d-bc98-1a30ec2a4d7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:37:04.888810 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04df264c-bd02-476d-bc98-1a30ec2a4d7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:04.890457 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04df264c-bd02-476d-bc98-1a30ec2a4d7e admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:04.894209 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8604744-ee87-4938-a78d-7dce339d89d9 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:04.901263 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd33d3e-5148-4016-9541-d8c554ace8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:04.902030 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd33d3e-5148-4016-9541-d8c554ace8d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:04.902282 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd33d3e-5148-4016-9541-d8c554ace8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:04.902563 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd33d3e-5148-4016-9541-d8c554ace8d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:04.903425 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd33d3e-5148-4016-9541-d8c554ace8d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:04.905590 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd33d3e-5148-4016-9541-d8c554ace8d0 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:04.920654 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1067/2147] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:37:04.925813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df264c-bd02-476d-bc98-1a30ec2a4d7e admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:04.930867 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1081/2148] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:04.943189 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd33d3e-5148-4016-9541-d8c554ace8d0 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:04.955998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df264c-bd02-476d-bc98-1a30ec2a4d7e admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:04.987719 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd33d3e-5148-4016-9541-d8c554ace8d0 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:04.991901 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc829a11-452f-45c1-a423-e9adaa186137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:04.992504 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc829a11-452f-45c1-a423-e9adaa186137 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:04.992788 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc829a11-452f-45c1-a423-e9adaa186137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:04.993060 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc829a11-452f-45c1-a423-e9adaa186137 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:04.998113 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc829a11-452f-45c1-a423-e9adaa186137 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:05.000093 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc829a11-452f-45c1-a423-e9adaa186137 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:05.007920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b0d2b-b33b-4eb9-a0e8-d589ca24217a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:05.009570 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b0d2b-b33b-4eb9-a0e8-d589ca24217a admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:05.009857 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b0d2b-b33b-4eb9-a0e8-d589ca24217a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:05.010136 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b0d2b-b33b-4eb9-a0e8-d589ca24217a admin admin] PATH_INFO: `/v3/projects/56ebc421702b4c35aff1bdbc620db716/users/95e2c9f732ec4f3e95f6e8c32e507300/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:05.039488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8b0d2b-b33b-4eb9-a0e8-d589ca24217a admin admin] RBAC: Authorizing `identity:create_grant(project_id=56ebc421702b4c35aff1bdbc620db716, user_id=95e2c9f732ec4f3e95f6e8c32e507300, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:05.042741 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8b0d2b-b33b-4eb9-a0e8-d589ca24217a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:05.090732 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:05.153447 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1082/2149] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:37:04 2022] PUT /identity/v3/projects/56ebc421702b4c35aff1bdbc620db716/users/95e2c9f732ec4f3e95f6e8c32e507300/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:05.154910 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5feedc-6a56-4f2d-a83a-66b24b4d76fa None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:05.155271 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5feedc-6a56-4f2d-a83a-66b24b4d76fa None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:05.155617 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5feedc-6a56-4f2d-a83a-66b24b4d76fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:05.179173 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1068/2150] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:37:04 2022] POST /identity/v3/projects => generated 375 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:05.185109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18185c35-0931-4a3d-9a0c-00a8164a4315 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:05.185408 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18185c35-0931-4a3d-9a0c-00a8164a4315 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:05.185685 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18185c35-0931-4a3d-9a0c-00a8164a4315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:05.232429 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-ec5feedc-6a56-4f2d-a83a-66b24b4d76fa None None] MFA Rules not processed for user `3dae32fa077840d9a053b6bef162615e`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:05.233868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ec5feedc-6a56-4f2d-a83a-66b24b4d76fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:05.265990 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-18185c35-0931-4a3d-9a0c-00a8164a4315 None None] MFA Rules not processed for user `95e2c9f732ec4f3e95f6e8c32e507300`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:05.268875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-18185c35-0931-4a3d-9a0c-00a8164a4315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:05.342396 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1083/2151] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:05 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:05.343439 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5feedc-6a56-4f2d-a83a-66b24b4d76fa None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:05.359937 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:05.385336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1069/2152] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:05.385336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18185c35-0931-4a3d-9a0c-00a8164a4315 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:05.385960 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-18185c35-0931-4a3d-9a0c-00a8164a4315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:05.437045 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5feedc-6a56-4f2d-a83a-66b24b4d76fa None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:05.443854 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e301fe-affb-405c-bce2-37ddb030a0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:05.444540 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e301fe-affb-405c-bce2-37ddb030a0f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:05.444818 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e301fe-affb-405c-bce2-37ddb030a0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:05.445120 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e301fe-affb-405c-bce2-37ddb030a0f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:05.445825 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e301fe-affb-405c-bce2-37ddb030a0f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:05.447833 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e301fe-affb-405c-bce2-37ddb030a0f5 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:05.473803 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18185c35-0931-4a3d-9a0c-00a8164a4315 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:05.479877 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a583d100-c19c-4de8-b03e-5e37a24fc1ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:05.480543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583d100-c19c-4de8-b03e-5e37a24fc1ea service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:05.480827 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583d100-c19c-4de8-b03e-5e37a24fc1ea service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:05.481253 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583d100-c19c-4de8-b03e-5e37a24fc1ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:05.483513 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a583d100-c19c-4de8-b03e-5e37a24fc1ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:05.530359 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1084/2153] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:37:05 2022] POST /identity/v3/users => generated 288 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:05.531341 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e301fe-affb-405c-bce2-37ddb030a0f5 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:05.545054 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e301fe-affb-405c-bce2-37ddb030a0f5 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:05.548822 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdfcc9-076b-462a-b9aa-30cd695309d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:05.549555 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdfcc9-076b-462a-b9aa-30cd695309d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:05.549814 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdfcc9-076b-462a-b9aa-30cd695309d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:05.550150 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcdfcc9-076b-462a-b9aa-30cd695309d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:05.550703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a583d100-c19c-4de8-b03e-5e37a24fc1ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:05.550869 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcdfcc9-076b-462a-b9aa-30cd695309d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:05.553159 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a583d100-c19c-4de8-b03e-5e37a24fc1ea service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:05.553343 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcdfcc9-076b-462a-b9aa-30cd695309d3 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:05.563723 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1070/2154] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:05.575997 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a583d100-c19c-4de8-b03e-5e37a24fc1ea service nova] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:05.577910 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a583d100-c19c-4de8-b03e-5e37a24fc1ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:05.591731 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a583d100-c19c-4de8-b03e-5e37a24fc1ea service nova] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:05.597690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6ddc5-3d27-4425-b7ce-bea5ce9a494c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:05.598595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6ddc5-3d27-4425-b7ce-bea5ce9a494c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:05.598904 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6ddc5-3d27-4425-b7ce-bea5ce9a494c admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:05.599223 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6ddc5-3d27-4425-b7ce-bea5ce9a494c admin admin] PATH_INFO: `/v3/projects/6ce9429917ee47fa94232afb35703316` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:05.600517 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6ddc5-3d27-4425-b7ce-bea5ce9a494c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ce9429917ee47fa94232afb35703316)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:05.602430 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6ddc5-3d27-4425-b7ce-bea5ce9a494c admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:05.646308 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1085/2155] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:37:05 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:05.646308 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdfcc9-076b-462a-b9aa-30cd695309d3 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:05.758749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c4e6ddc5-3d27-4425-b7ce-bea5ce9a494c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ce9429917ee47fa94232afb35703316', 'request_id': 'req-c4e6ddc5-3d27-4425-b7ce-bea5ce9a494c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:05.760580 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcdfcc9-076b-462a-b9aa-30cd695309d3 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:05.821211 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c4e6ddc5-3d27-4425-b7ce-bea5ce9a494c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ce9429917ee47fa94232afb35703316', 'request_id': 'req-c4e6ddc5-3d27-4425-b7ce-bea5ce9a494c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:05.829823 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c4e6ddc5-3d27-4425-b7ce-bea5ce9a494c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ce9429917ee47fa94232afb35703316', 'request_id': 'req-c4e6ddc5-3d27-4425-b7ce-bea5ce9a494c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:05.841030 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44a9a1-6703-42bc-9a28-a60d91567178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:05.841721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44a9a1-6703-42bc-9a28-a60d91567178 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:05.842008 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44a9a1-6703-42bc-9a28-a60d91567178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:05.844644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44a9a1-6703-42bc-9a28-a60d91567178 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:05.845488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b44a9a1-6703-42bc-9a28-a60d91567178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:05.848312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b44a9a1-6703-42bc-9a28-a60d91567178 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:05.853749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c4e6ddc5-3d27-4425-b7ce-bea5ce9a494c admin admin] The token cache is being invalidate because project 6ce9429917ee47fa94232afb35703316 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:37:05.856629 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c4e6ddc5-3d27-4425-b7ce-bea5ce9a494c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:05.857863 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c4e6ddc5-3d27-4425-b7ce-bea5ce9a494c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:05.863276 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1071/2156] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:37:05 2022] DELETE /identity/v3/projects/6ce9429917ee47fa94232afb35703316 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:05.864466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6ddc5-3d27-4425-b7ce-bea5ce9a494c admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:05.869832 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c4e6ddc5-3d27-4425-b7ce-bea5ce9a494c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:05.872352 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1086/2157] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:05.874069 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ef3e5-fae0-4197-8115-ca9522b12674 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:05.874435 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ef3e5-fae0-4197-8115-ca9522b12674 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:05.874720 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ef3e5-fae0-4197-8115-ca9522b12674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:05.899850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6ddc5-3d27-4425-b7ce-bea5ce9a494c admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:05.900403 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-e63ef3e5-fae0-4197-8115-ca9522b12674 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:05.902121 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e63ef3e5-fae0-4197-8115-ca9522b12674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:05.909395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b15d76-c2f7-407d-943e-c5c136f3b93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:05.910629 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b15d76-c2f7-407d-943e-c5c136f3b93f admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:05.911137 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b15d76-c2f7-407d-943e-c5c136f3b93f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:05.911585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b15d76-c2f7-407d-943e-c5c136f3b93f admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:05.913168 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b15d76-c2f7-407d-943e-c5c136f3b93f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:05.915310 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b15d76-c2f7-407d-943e-c5c136f3b93f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:05.922396 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1087/2158] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:05.923427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ef3e5-fae0-4197-8115-ca9522b12674 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:05.924881 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e63ef3e5-fae0-4197-8115-ca9522b12674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:05.942447 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ef3e5-fae0-4197-8115-ca9522b12674 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:05.948834 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c1366-4a80-4856-b940-73e4a5f7626b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:05.949518 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c1366-4a80-4856-b940-73e4a5f7626b admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:05.949805 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c1366-4a80-4856-b940-73e4a5f7626b admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:05.950084 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c1366-4a80-4856-b940-73e4a5f7626b admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:05.950818 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713c1366-4a80-4856-b940-73e4a5f7626b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:05.953559 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713c1366-4a80-4856-b940-73e4a5f7626b admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:05.964733 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1088/2159] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:05.966018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8e5f5-9700-4c50-adcf-9e8480f6d215 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:05.966364 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8e5f5-9700-4c50-adcf-9e8480f6d215 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:05.966598 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8e5f5-9700-4c50-adcf-9e8480f6d215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:06.001794 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-e6a8e5f5-9700-4c50-adcf-9e8480f6d215 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:06.003638 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e6a8e5f5-9700-4c50-adcf-9e8480f6d215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:06.009999 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1072/2160] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:37:05 2022] POST /identity/v3/users => generated 299 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:06.010854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b15d76-c2f7-407d-943e-c5c136f3b93f admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:06.015486 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1089/2161] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:06.016532 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a8e5f5-9700-4c50-adcf-9e8480f6d215 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:06.028753 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b15d76-c2f7-407d-943e-c5c136f3b93f admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:06.033749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3483c-0f7d-41c6-acc1-3f6b3a8b22d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:06.034658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3483c-0f7d-41c6-acc1-3f6b3a8b22d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:06.035052 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3483c-0f7d-41c6-acc1-3f6b3a8b22d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:06.035360 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3483c-0f7d-41c6-acc1-3f6b3a8b22d4 admin admin] PATH_INFO: `/v3/projects/62f595fe610a403993ae86f3899eb494/users/58205a78eae64db4a5f98a0726813728/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:06.035510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a8e5f5-9700-4c50-adcf-9e8480f6d215 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:06.041649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b99bfc-f4d3-404c-a95b-3cb257331e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:06.042446 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b99bfc-f4d3-404c-a95b-3cb257331e2e admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:06.042992 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b99bfc-f4d3-404c-a95b-3cb257331e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:06.043399 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b99bfc-f4d3-404c-a95b-3cb257331e2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:06.044353 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b99bfc-f4d3-404c-a95b-3cb257331e2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:06.046322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b99bfc-f4d3-404c-a95b-3cb257331e2e admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:06.051662 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3483c-0f7d-41c6-acc1-3f6b3a8b22d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62f595fe610a403993ae86f3899eb494, user_id=58205a78eae64db4a5f98a0726813728, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:06.054678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3483c-0f7d-41c6-acc1-3f6b3a8b22d4 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:06.061767 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1090/2162] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:06.061767 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b99bfc-f4d3-404c-a95b-3cb257331e2e admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:06.069911 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b99bfc-f4d3-404c-a95b-3cb257331e2e admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:06.076177 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69990288-1a0f-4eda-9adf-0ca088e6e91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:06.076842 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69990288-1a0f-4eda-9adf-0ca088e6e91c admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:06.077389 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69990288-1a0f-4eda-9adf-0ca088e6e91c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:06.080625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69990288-1a0f-4eda-9adf-0ca088e6e91c admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:06.080625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69990288-1a0f-4eda-9adf-0ca088e6e91c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:37:06.080625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69990288-1a0f-4eda-9adf-0ca088e6e91c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:06.080625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69990288-1a0f-4eda-9adf-0ca088e6e91c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:06.093780 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1091/2163] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:37:06.094672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69990288-1a0f-4eda-9adf-0ca088e6e91c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:06.106363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69990288-1a0f-4eda-9adf-0ca088e6e91c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:06.111249 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e3376a-5f25-46a4-af65-88c2f1722d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:06.112179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3376a-5f25-46a4-af65-88c2f1722d8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:06.112462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3376a-5f25-46a4-af65-88c2f1722d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:06.112744 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e3376a-5f25-46a4-af65-88c2f1722d8a admin admin] PATH_INFO: `/v3/projects/d30f186e29664bd482f3e17b5dc2f4fe/users/256f9ca6ebca4e95b3903777cf30a88d/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:06.129422 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e3376a-5f25-46a4-af65-88c2f1722d8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d30f186e29664bd482f3e17b5dc2f4fe, user_id=256f9ca6ebca4e95b3903777cf30a88d, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:06.132320 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e3376a-5f25-46a4-af65-88c2f1722d8a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:06.150884 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1073/2164] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:37:06 2022] PUT /identity/v3/projects/62f595fe610a403993ae86f3899eb494/users/58205a78eae64db4a5f98a0726813728/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:06.151792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3483c-0f7d-41c6-acc1-3f6b3a8b22d4 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:06.212557 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3483c-0f7d-41c6-acc1-3f6b3a8b22d4 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:06.217373 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c523d-dd61-4610-b874-832b61f3b10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:06.218029 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c523d-dd61-4610-b874-832b61f3b10f admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:06.218314 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c523d-dd61-4610-b874-832b61f3b10f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:06.218620 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c523d-dd61-4610-b874-832b61f3b10f admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:06.219353 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c523d-dd61-4610-b874-832b61f3b10f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:06.221734 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c523d-dd61-4610-b874-832b61f3b10f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:06.262817 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1092/2165] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:37:06 2022] PUT /identity/v3/projects/d30f186e29664bd482f3e17b5dc2f4fe/users/256f9ca6ebca4e95b3903777cf30a88d/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:06.264802 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5f6d9-5bb5-4048-91b0-d0cc4b6196a9 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:06.265398 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5f6d9-5bb5-4048-91b0-d0cc4b6196a9 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:06.265722 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd5f6d9-5bb5-4048-91b0-d0cc4b6196a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:06.297141 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-4dd5f6d9-5bb5-4048-91b0-d0cc4b6196a9 None None] MFA Rules not processed for user `58205a78eae64db4a5f98a0726813728`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:06.298628 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4dd5f6d9-5bb5-4048-91b0-d0cc4b6196a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:06.350583 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1074/2166] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:37:06 2022] POST /identity/v3/projects => generated 361 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:06.350583 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c523d-dd61-4610-b874-832b61f3b10f admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:06.381965 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1093/2167] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:06.382666 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5f6d9-5bb5-4048-91b0-d0cc4b6196a9 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:06.420286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c523d-dd61-4610-b874-832b61f3b10f admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:06.425174 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fab80-410f-44bb-895b-b4766d81275b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:06.425847 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fab80-410f-44bb-895b-b4766d81275b admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:06.426130 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fab80-410f-44bb-895b-b4766d81275b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:06.426404 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fab80-410f-44bb-895b-b4766d81275b admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:06.427177 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26fab80-410f-44bb-895b-b4766d81275b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:06.429656 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26fab80-410f-44bb-895b-b4766d81275b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:06.440058 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1075/2168] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:06.441041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fab80-410f-44bb-895b-b4766d81275b admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:06.453623 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fab80-410f-44bb-895b-b4766d81275b admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:06.459846 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d78f95a-00c7-4442-a001-f33048f1cca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:06.460560 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78f95a-00c7-4442-a001-f33048f1cca2 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:06.460847 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78f95a-00c7-4442-a001-f33048f1cca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:06.461123 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d78f95a-00c7-4442-a001-f33048f1cca2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:06.461824 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d78f95a-00c7-4442-a001-f33048f1cca2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:06.463864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d78f95a-00c7-4442-a001-f33048f1cca2 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:06.464168 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd5f6d9-5bb5-4048-91b0-d0cc4b6196a9 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:06.471446 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56699d54-b213-45d9-bb1b-3bf999e12de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:06.472217 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56699d54-b213-45d9-bb1b-3bf999e12de6 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:06.472574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56699d54-b213-45d9-bb1b-3bf999e12de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:06.472930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56699d54-b213-45d9-bb1b-3bf999e12de6 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:06.473709 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56699d54-b213-45d9-bb1b-3bf999e12de6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:06.482391 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56699d54-b213-45d9-bb1b-3bf999e12de6 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:06.582775 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1094/2169] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:37:06 2022] POST /identity/v3/users => generated 281 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:06.582775 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56699d54-b213-45d9-bb1b-3bf999e12de6 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:06.588557 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1076/2170] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:37:06 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:06.588557 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d78f95a-00c7-4442-a001-f33048f1cca2 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:06.608263 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56699d54-b213-45d9-bb1b-3bf999e12de6 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:06.693517 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d78f95a-00c7-4442-a001-f33048f1cca2 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:06.699302 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288263fa-c22d-4691-b1cd-6d1d2bb9e216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:06.700786 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288263fa-c22d-4691-b1cd-6d1d2bb9e216 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:06.701180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288263fa-c22d-4691-b1cd-6d1d2bb9e216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:06.702245 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288263fa-c22d-4691-b1cd-6d1d2bb9e216 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:06.703600 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288263fa-c22d-4691-b1cd-6d1d2bb9e216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:06.706497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288263fa-c22d-4691-b1cd-6d1d2bb9e216 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:06.714629 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:06.730302 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1077/2171] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:06.731375 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288263fa-c22d-4691-b1cd-6d1d2bb9e216 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:06.744636 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f570f-0e46-46f2-95a0-a3d54415cfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:06.751467 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f570f-0e46-46f2-95a0-a3d54415cfa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:06.751759 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f570f-0e46-46f2-95a0-a3d54415cfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:06.752043 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f570f-0e46-46f2-95a0-a3d54415cfa1 admin admin] PATH_INFO: `/v3/projects/d30f186e29664bd482f3e17b5dc2f4fe/users/256f9ca6ebca4e95b3903777cf30a88d/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:06.752445 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288263fa-c22d-4691-b1cd-6d1d2bb9e216 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:06.757913 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de78472-eac2-40a8-9a04-ad9b0438bdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:06.758530 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f570f-0e46-46f2-95a0-a3d54415cfa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d30f186e29664bd482f3e17b5dc2f4fe, user_id=256f9ca6ebca4e95b3903777cf30a88d, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:06.758641 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de78472-eac2-40a8-9a04-ad9b0438bdf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:06.758910 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de78472-eac2-40a8-9a04-ad9b0438bdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:06.759176 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de78472-eac2-40a8-9a04-ad9b0438bdf5 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:06.759959 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de78472-eac2-40a8-9a04-ad9b0438bdf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:06.761277 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356f570f-0e46-46f2-95a0-a3d54415cfa1 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:06.761940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de78472-eac2-40a8-9a04-ad9b0438bdf5 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:06.772663 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1095/2172] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:37:06 2022] PUT /identity/v3/projects/d30f186e29664bd482f3e17b5dc2f4fe/users/256f9ca6ebca4e95b3903777cf30a88d/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:06.773501 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f570f-0e46-46f2-95a0-a3d54415cfa1 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:06.774393 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:06.839269 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356f570f-0e46-46f2-95a0-a3d54415cfa1 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:06.844050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d564e-528c-4d95-b1e5-6be414a94f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:06.844715 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d564e-528c-4d95-b1e5-6be414a94f4a admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:06.844997 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d564e-528c-4d95-b1e5-6be414a94f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:06.845273 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d564e-528c-4d95-b1e5-6be414a94f4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:06.846002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8d564e-528c-4d95-b1e5-6be414a94f4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:06.847641 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8d564e-528c-4d95-b1e5-6be414a94f4a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:06.858263 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1096/2173] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:06.859588 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883e7bb-4f80-433a-81e0-b5088e6c532b None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:06.859887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883e7bb-4f80-433a-81e0-b5088e6c532b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:06.860140 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1883e7bb-4f80-433a-81e0-b5088e6c532b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:06.888690 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-1883e7bb-4f80-433a-81e0-b5088e6c532b None None] MFA Rules not processed for user `256f9ca6ebca4e95b3903777cf30a88d`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:06.890225 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1883e7bb-4f80-433a-81e0-b5088e6c532b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:06.892555 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1078/2174] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:37:06 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:06.893466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de78472-eac2-40a8-9a04-ad9b0438bdf5 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:06.910357 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de78472-eac2-40a8-9a04-ad9b0438bdf5 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:06.918633 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e11ab-9e76-4f3d-841e-1258ecbc1118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:06.920848 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e11ab-9e76-4f3d-841e-1258ecbc1118 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:06.921779 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e11ab-9e76-4f3d-841e-1258ecbc1118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:06.922727 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e11ab-9e76-4f3d-841e-1258ecbc1118 admin admin] PATH_INFO: `/v3/projects/d2d2d7f94dc544f5aa4da494607d397c/users/d906d61ed2394c47bf69cd5286ba45e3/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:06.953446 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e11ab-9e76-4f3d-841e-1258ecbc1118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2d2d7f94dc544f5aa4da494607d397c, user_id=d906d61ed2394c47bf69cd5286ba45e3, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:06.956886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e11ab-9e76-4f3d-841e-1258ecbc1118 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:07.041746 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1097/2175] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:06 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:07.043048 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883e7bb-4f80-433a-81e0-b5088e6c532b None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:07.074994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1079/2176] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:37:06 2022] PUT /identity/v3/projects/d2d2d7f94dc544f5aa4da494607d397c/users/d906d61ed2394c47bf69cd5286ba45e3/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:07.074994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e11ab-9e76-4f3d-841e-1258ecbc1118 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:07.076572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1883e7bb-4f80-433a-81e0-b5088e6c532b None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:07.095051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e76e11ab-9e76-4f3d-841e-1258ecbc1118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:07.201756 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e11ab-9e76-4f3d-841e-1258ecbc1118 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:07.206681 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0867b169-7954-4015-8356-c449d04d8654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:07.207468 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867b169-7954-4015-8356-c449d04d8654 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:07.207750 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867b169-7954-4015-8356-c449d04d8654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:07.208073 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0867b169-7954-4015-8356-c449d04d8654 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:07.209459 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867b169-7954-4015-8356-c449d04d8654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:07.211992 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccec7a7-367a-4899-8788-9a8204b8d17f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:07.212661 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccec7a7-367a-4899-8788-9a8204b8d17f service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:07.212994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccec7a7-367a-4899-8788-9a8204b8d17f service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:07.213411 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccec7a7-367a-4899-8788-9a8204b8d17f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:07.219203 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0867b169-7954-4015-8356-c449d04d8654 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:07.235588 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1098/2177] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:07.240924 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e1b5f-3ce6-416e-b3e5-3f334bc37bd6 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:07.241218 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e1b5f-3ce6-416e-b3e5-3f334bc37bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:07.241501 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e1b5f-3ce6-416e-b3e5-3f334bc37bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:07.292378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-de8e1b5f-3ce6-416e-b3e5-3f334bc37bd6 None None] MFA Rules not processed for user `d906d61ed2394c47bf69cd5286ba45e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:07.293865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-de8e1b5f-3ce6-416e-b3e5-3f334bc37bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:07.322673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccec7a7-367a-4899-8788-9a8204b8d17f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:07.324606 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccec7a7-367a-4899-8788-9a8204b8d17f service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:07.351387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1080/2178] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:07.351387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccec7a7-367a-4899-8788-9a8204b8d17f service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:07.363457 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccec7a7-367a-4899-8788-9a8204b8d17f service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:07.372578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e00e1-89a3-4992-be94-f6e5180fb968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:07.373882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e00e1-89a3-4992-be94-f6e5180fb968 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:07.374232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e00e1-89a3-4992-be94-f6e5180fb968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:07.374855 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e00e1-89a3-4992-be94-f6e5180fb968 admin admin] PATH_INFO: `/v3/projects/7156a444d77b40d5b56b6b62122ce14e/users/eebc19aad8b84a7396835091a6e00a64/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:07.405213 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e00e1-89a3-4992-be94-f6e5180fb968 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7156a444d77b40d5b56b6b62122ce14e, user_id=eebc19aad8b84a7396835091a6e00a64, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:07.406261 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1099/2179] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:07 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:07.409150 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e1b5f-3ce6-416e-b3e5-3f334bc37bd6 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:07.417235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e00e1-89a3-4992-be94-f6e5180fb968 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:07.441874 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e1b5f-3ce6-416e-b3e5-3f334bc37bd6 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:07.447845 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144e5df-ec45-40ba-b5c6-4ba06261136a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:07.448602 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144e5df-ec45-40ba-b5c6-4ba06261136a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:07.448910 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144e5df-ec45-40ba-b5c6-4ba06261136a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:07.449480 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144e5df-ec45-40ba-b5c6-4ba06261136a admin admin] PATH_INFO: `/v3/users/95e2c9f732ec4f3e95f6e8c32e507300` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:07.450553 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7144e5df-ec45-40ba-b5c6-4ba06261136a admin admin] RBAC: Authorizing `identity:delete_user(user_id=95e2c9f732ec4f3e95f6e8c32e507300)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:07.452815 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7144e5df-ec45-40ba-b5c6-4ba06261136a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:07.487576 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1081/2180] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:37:07 2022] PUT /identity/v3/projects/7156a444d77b40d5b56b6b62122ce14e/users/eebc19aad8b84a7396835091a6e00a64/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:07.491717 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e00e1-89a3-4992-be94-f6e5180fb968 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:07.491717 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-348e00e1-89a3-4992-be94-f6e5180fb968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:07.591701 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e00e1-89a3-4992-be94-f6e5180fb968 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:07.608798 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1f34f-ce01-4f99-9156-da7f08797fcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:07.609600 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1f34f-ce01-4f99-9156-da7f08797fcd service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:07.610147 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1f34f-ce01-4f99-9156-da7f08797fcd service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:07.610560 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1f34f-ce01-4f99-9156-da7f08797fcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:07.658946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7144e5df-ec45-40ba-b5c6-4ba06261136a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95e2c9f732ec4f3e95f6e8c32e507300'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:07.688274 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1f34f-ce01-4f99-9156-da7f08797fcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:07.689683 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1f34f-ce01-4f99-9156-da7f08797fcd service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:07.705129 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1082/2181] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:07 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:07.705129 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1f34f-ce01-4f99-9156-da7f08797fcd service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:07.738716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7144e5df-ec45-40ba-b5c6-4ba06261136a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95e2c9f732ec4f3e95f6e8c32e507300'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:07.760396 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7144e5df-ec45-40ba-b5c6-4ba06261136a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95e2c9f732ec4f3e95f6e8c32e507300'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:07.761205 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7144e5df-ec45-40ba-b5c6-4ba06261136a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95e2c9f732ec4f3e95f6e8c32e507300'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:07.770223 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7144e5df-ec45-40ba-b5c6-4ba06261136a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95e2c9f732ec4f3e95f6e8c32e507300'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:07.782192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1100/2182] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:37:07 2022] DELETE /identity/v3/users/95e2c9f732ec4f3e95f6e8c32e507300 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:07.782192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144e5df-ec45-40ba-b5c6-4ba06261136a admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:07.782472 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-7144e5df-ec45-40ba-b5c6-4ba06261136a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:07.790099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1f34f-ce01-4f99-9156-da7f08797fcd service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:07.885504 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613c021-ac9e-4d01-88bf-370518532d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:07.886216 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613c021-ac9e-4d01-88bf-370518532d55 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:07.886511 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613c021-ac9e-4d01-88bf-370518532d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:07.887486 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613c021-ac9e-4d01-88bf-370518532d55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:07.888227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613c021-ac9e-4d01-88bf-370518532d55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:07.890083 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613c021-ac9e-4d01-88bf-370518532d55 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:07.904998 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144e5df-ec45-40ba-b5c6-4ba06261136a admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:07.915133 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1083/2183] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:07 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:07.920916 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2313d2-5b39-457b-8bf2-1b8a75f7a121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:07.920916 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2313d2-5b39-457b-8bf2-1b8a75f7a121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:07.920916 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2313d2-5b39-457b-8bf2-1b8a75f7a121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:07.920916 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2313d2-5b39-457b-8bf2-1b8a75f7a121 admin admin] PATH_INFO: `/v3/users/d906d61ed2394c47bf69cd5286ba45e3` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:07.921534 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc59cc-a909-4c2d-b3d7-bfeb7336ecba None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:07.921534 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc59cc-a909-4c2d-b3d7-bfeb7336ecba None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:07.921534 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc59cc-a909-4c2d-b3d7-bfeb7336ecba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:07.922763 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2313d2-5b39-457b-8bf2-1b8a75f7a121 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d906d61ed2394c47bf69cd5286ba45e3)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:07.924802 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2313d2-5b39-457b-8bf2-1b8a75f7a121 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:07.945114 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-58dc59cc-a909-4c2d-b3d7-bfeb7336ecba None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:07.953437 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-58dc59cc-a909-4c2d-b3d7-bfeb7336ecba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:07.955807 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1084/2184] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:07.961606 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc59cc-a909-4c2d-b3d7-bfeb7336ecba None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:07.963147 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-58dc59cc-a909-4c2d-b3d7-bfeb7336ecba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:07.977237 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc59cc-a909-4c2d-b3d7-bfeb7336ecba None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:07.982214 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d6d69-ddf6-4bb2-afbc-1fdebbf5adc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:07.983161 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d6d69-ddf6-4bb2-afbc-1fdebbf5adc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:07.983716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d6d69-ddf6-4bb2-afbc-1fdebbf5adc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:07.983716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d6d69-ddf6-4bb2-afbc-1fdebbf5adc2 admin admin] PATH_INFO: `/v3/projects/7156a444d77b40d5b56b6b62122ce14e/users/eebc19aad8b84a7396835091a6e00a64/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:07.985687 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d6d69-ddf6-4bb2-afbc-1fdebbf5adc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7156a444d77b40d5b56b6b62122ce14e, user_id=eebc19aad8b84a7396835091a6e00a64, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:07.988599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d6d69-ddf6-4bb2-afbc-1fdebbf5adc2 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:07.999781 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1085/2185] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:37:07 2022] PUT /identity/v3/projects/7156a444d77b40d5b56b6b62122ce14e/users/eebc19aad8b84a7396835091a6e00a64/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:08.001216 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d6d69-ddf6-4bb2-afbc-1fdebbf5adc2 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:08.006206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-536d6d69-ddf6-4bb2-afbc-1fdebbf5adc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:08.067283 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d6d69-ddf6-4bb2-afbc-1fdebbf5adc2 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:08.076583 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02181f9a-2416-4531-8457-c9926100ad7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:08.076795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02181f9a-2416-4531-8457-c9926100ad7d service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:08.077328 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02181f9a-2416-4531-8457-c9926100ad7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:08.078230 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02181f9a-2416-4531-8457-c9926100ad7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:08.148031 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02181f9a-2416-4531-8457-c9926100ad7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:08.149544 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02181f9a-2416-4531-8457-c9926100ad7d service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:08.160033 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1086/2186] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:08.161352 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c5c35-3a2a-4981-a608-480f7adc5d1f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:08.161648 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c5c35-3a2a-4981-a608-480f7adc5d1f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:08.161948 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c5c35-3a2a-4981-a608-480f7adc5d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:08.178710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0a2313d2-5b39-457b-8bf2-1b8a75f7a121 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd906d61ed2394c47bf69cd5286ba45e3'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:08.192072 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-042c5c35-3a2a-4981-a608-480f7adc5d1f None None] MFA Rules not processed for user `eebc19aad8b84a7396835091a6e00a64`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:08.194170 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-042c5c35-3a2a-4981-a608-480f7adc5d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:08.304550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0a2313d2-5b39-457b-8bf2-1b8a75f7a121 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd906d61ed2394c47bf69cd5286ba45e3'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:08.320541 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1087/2187] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:08 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:08.332428 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0a2313d2-5b39-457b-8bf2-1b8a75f7a121 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd906d61ed2394c47bf69cd5286ba45e3'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:08.334530 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0a2313d2-5b39-457b-8bf2-1b8a75f7a121 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd906d61ed2394c47bf69cd5286ba45e3'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:08.342548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c5c35-3a2a-4981-a608-480f7adc5d1f None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:08.344402 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-042c5c35-3a2a-4981-a608-480f7adc5d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:08.353985 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0a2313d2-5b39-457b-8bf2-1b8a75f7a121 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd906d61ed2394c47bf69cd5286ba45e3'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:08.368943 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1101/2188] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:37:07 2022] DELETE /identity/v3/users/d906d61ed2394c47bf69cd5286ba45e3 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:08.373383 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745209d-f6a5-4372-a62c-9c60a4cd2a41 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:08.373712 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745209d-f6a5-4372-a62c-9c60a4cd2a41 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:08.374000 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745209d-f6a5-4372-a62c-9c60a4cd2a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:08.415562 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-0745209d-f6a5-4372-a62c-9c60a4cd2a41 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:08.417788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-0745209d-f6a5-4372-a62c-9c60a4cd2a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:08.511195 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1102/2189] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:37:08.521769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c5c35-3a2a-4981-a608-480f7adc5d1f None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:08.531028 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745209d-f6a5-4372-a62c-9c60a4cd2a41 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:08.532422 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-0745209d-f6a5-4372-a62c-9c60a4cd2a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:08.621780 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94790a8b-441f-4fcc-b465-e85b573fbea7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:08.622547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94790a8b-441f-4fcc-b465-e85b573fbea7 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:08.623250 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94790a8b-441f-4fcc-b465-e85b573fbea7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:08.624154 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94790a8b-441f-4fcc-b465-e85b573fbea7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:08.628306 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-94790a8b-441f-4fcc-b465-e85b573fbea7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:08.678818 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745209d-f6a5-4372-a62c-9c60a4cd2a41 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:08.698686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc14311-9107-41d9-9293-a08ef90e472b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:08.699497 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc14311-9107-41d9-9293-a08ef90e472b service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:08.704482 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc14311-9107-41d9-9293-a08ef90e472b service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:08.707627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc14311-9107-41d9-9293-a08ef90e472b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:08.732000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:08.744359 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc14311-9107-41d9-9293-a08ef90e472b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:08.749605 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc14311-9107-41d9-9293-a08ef90e472b service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:08.771781 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1103/2190] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:37:08.772210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94790a8b-441f-4fcc-b465-e85b573fbea7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:08.775091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94790a8b-441f-4fcc-b465-e85b573fbea7 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:08.797900 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1088/2191] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:09.408349 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534aa90-cf2a-4c68-a229-03c87540ca87 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:09.408562 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534aa90-cf2a-4c68-a229-03c87540ca87 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:09.408832 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534aa90-cf2a-4c68-a229-03c87540ca87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:09.433036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-d534aa90-cf2a-4c68-a229-03c87540ca87 None None] MFA Rules not processed for user `9001a4b10d544226b3b1453b06883db4`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:09.434243 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d534aa90-cf2a-4c68-a229-03c87540ca87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:09.487426 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94790a8b-441f-4fcc-b465-e85b573fbea7 service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:09.488985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-94790a8b-441f-4fcc-b465-e85b573fbea7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:09.493808 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1104/2192] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:09.526093 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94790a8b-441f-4fcc-b465-e85b573fbea7 service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:09.531455 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77db63e-14d0-455e-b43c-351aeceeb57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:09.536000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77db63e-14d0-455e-b43c-351aeceeb57d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:09.536288 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77db63e-14d0-455e-b43c-351aeceeb57d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:09.536733 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77db63e-14d0-455e-b43c-351aeceeb57d admin admin] PATH_INFO: `/v3/projects/56ebc421702b4c35aff1bdbc620db716` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:09.538667 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77db63e-14d0-455e-b43c-351aeceeb57d admin admin] RBAC: Authorizing `identity:delete_project(project_id=56ebc421702b4c35aff1bdbc620db716)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:09.541054 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77db63e-14d0-455e-b43c-351aeceeb57d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:09.721043 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e77db63e-14d0-455e-b43c-351aeceeb57d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56ebc421702b4c35aff1bdbc620db716', 'request_id': 'req-e77db63e-14d0-455e-b43c-351aeceeb57d'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:09.847743 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e77db63e-14d0-455e-b43c-351aeceeb57d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56ebc421702b4c35aff1bdbc620db716', 'request_id': 'req-e77db63e-14d0-455e-b43c-351aeceeb57d'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:09.860210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e77db63e-14d0-455e-b43c-351aeceeb57d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56ebc421702b4c35aff1bdbc620db716', 'request_id': 'req-e77db63e-14d0-455e-b43c-351aeceeb57d'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:09.879461 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e77db63e-14d0-455e-b43c-351aeceeb57d admin admin] The token cache is being invalidate because project 56ebc421702b4c35aff1bdbc620db716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:37:09.887077 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:09.889002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e77db63e-14d0-455e-b43c-351aeceeb57d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:09.890008 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e77db63e-14d0-455e-b43c-351aeceeb57d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:09.895250 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1089/2193] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:37:09 2022] DELETE /identity/v3/projects/56ebc421702b4c35aff1bdbc620db716 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:37:09.896721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d534aa90-cf2a-4c68-a229-03c87540ca87 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:09.918215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d534aa90-cf2a-4c68-a229-03c87540ca87 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:10.036032 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c26421a-cf76-4433-ba31-264ec3b1ed9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:10.037020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26421a-cf76-4433-ba31-264ec3b1ed9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:10.037780 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26421a-cf76-4433-ba31-264ec3b1ed9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:10.038057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c26421a-cf76-4433-ba31-264ec3b1ed9e admin admin] PATH_INFO: `/v3/users/256f9ca6ebca4e95b3903777cf30a88d` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:10.039606 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c26421a-cf76-4433-ba31-264ec3b1ed9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=256f9ca6ebca4e95b3903777cf30a88d)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:10.044259 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c26421a-cf76-4433-ba31-264ec3b1ed9e admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:10.251130 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77db63e-14d0-455e-b43c-351aeceeb57d admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:10.252684 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e77db63e-14d0-455e-b43c-351aeceeb57d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:10.268123 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77db63e-14d0-455e-b43c-351aeceeb57d admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:10.273877 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4da558-d981-44c5-b8e4-b9ecd6f5a6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:10.275369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4da558-d981-44c5-b8e4-b9ecd6f5a6ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:10.275369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4da558-d981-44c5-b8e4-b9ecd6f5a6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:10.275753 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4da558-d981-44c5-b8e4-b9ecd6f5a6ee admin admin] PATH_INFO: `/v3/projects/d2d2d7f94dc544f5aa4da494607d397c` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:10.277987 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4da558-d981-44c5-b8e4-b9ecd6f5a6ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2d2d7f94dc544f5aa4da494607d397c)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:10.281084 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4da558-d981-44c5-b8e4-b9ecd6f5a6ee admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:10.310057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7c26421a-cf76-4433-ba31-264ec3b1ed9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '256f9ca6ebca4e95b3903777cf30a88d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:10.421157 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7c26421a-cf76-4433-ba31-264ec3b1ed9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '256f9ca6ebca4e95b3903777cf30a88d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:10.446958 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7c26421a-cf76-4433-ba31-264ec3b1ed9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '256f9ca6ebca4e95b3903777cf30a88d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:10.448250 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7c26421a-cf76-4433-ba31-264ec3b1ed9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '256f9ca6ebca4e95b3903777cf30a88d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:10.452942 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-eb4da558-d981-44c5-b8e4-b9ecd6f5a6ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2d2d7f94dc544f5aa4da494607d397c', 'request_id': 'req-eb4da558-d981-44c5-b8e4-b9ecd6f5a6ee'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:10.459772 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7c26421a-cf76-4433-ba31-264ec3b1ed9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '256f9ca6ebca4e95b3903777cf30a88d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:10.463424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1105/2194] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:37:09 2022] DELETE /identity/v3/users/256f9ca6ebca4e95b3903777cf30a88d => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:10.558199 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-eb4da558-d981-44c5-b8e4-b9ecd6f5a6ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2d2d7f94dc544f5aa4da494607d397c', 'request_id': 'req-eb4da558-d981-44c5-b8e4-b9ecd6f5a6ee'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:10.568820 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-eb4da558-d981-44c5-b8e4-b9ecd6f5a6ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2d2d7f94dc544f5aa4da494607d397c', 'request_id': 'req-eb4da558-d981-44c5-b8e4-b9ecd6f5a6ee'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:10.579820 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-eb4da558-d981-44c5-b8e4-b9ecd6f5a6ee admin admin] The token cache is being invalidate because project d2d2d7f94dc544f5aa4da494607d397c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:37:10.580058 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-eb4da558-d981-44c5-b8e4-b9ecd6f5a6ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:10.580730 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-eb4da558-d981-44c5-b8e4-b9ecd6f5a6ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:10.582367 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1090/2195] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:37:10 2022] DELETE /identity/v3/projects/d2d2d7f94dc544f5aa4da494607d397c => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:37:10.587577 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595e893-8695-45d9-9593-f412f60e4106 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:10.587874 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595e893-8695-45d9-9593-f412f60e4106 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:10.588212 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595e893-8695-45d9-9593-f412f60e4106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:10.606346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-9595e893-8695-45d9-9593-f412f60e4106 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:10.607579 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-9595e893-8695-45d9-9593-f412f60e4106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:10.667701 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1106/2196] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:10.676773 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149ea64-91a5-4c43-8812-80aff1ccffdc None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:10.676940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149ea64-91a5-4c43-8812-80aff1ccffdc None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:10.677297 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149ea64-91a5-4c43-8812-80aff1ccffdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:10.699912 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-f149ea64-91a5-4c43-8812-80aff1ccffdc None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:10.701090 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f149ea64-91a5-4c43-8812-80aff1ccffdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:10.710247 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1091/2197] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:37:10.714247 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595e893-8695-45d9-9593-f412f60e4106 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:10.728139 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595e893-8695-45d9-9593-f412f60e4106 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:10.732948 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc68cdb-6cf4-41ef-bf35-71bed2e7f5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:10.733619 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc68cdb-6cf4-41ef-bf35-71bed2e7f5cf admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:10.733949 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc68cdb-6cf4-41ef-bf35-71bed2e7f5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:10.734371 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc68cdb-6cf4-41ef-bf35-71bed2e7f5cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:10.735000 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc68cdb-6cf4-41ef-bf35-71bed2e7f5cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:37:10.735382 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc68cdb-6cf4-41ef-bf35-71bed2e7f5cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:10.737293 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc68cdb-6cf4-41ef-bf35-71bed2e7f5cf admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:10.750458 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1107/2198] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:37:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:37:10.756531 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149ea64-91a5-4c43-8812-80aff1ccffdc None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:10.772515 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149ea64-91a5-4c43-8812-80aff1ccffdc None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:10.777665 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9db502-74ca-4ba2-b45c-c373f2cc19e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:10.778359 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9db502-74ca-4ba2-b45c-c373f2cc19e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:10.778647 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9db502-74ca-4ba2-b45c-c373f2cc19e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:10.778958 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9db502-74ca-4ba2-b45c-c373f2cc19e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:10.791002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9db502-74ca-4ba2-b45c-c373f2cc19e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:10.791002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9db502-74ca-4ba2-b45c-c373f2cc19e5 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:10.871141 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1092/2199] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:37:10 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:37:10.875400 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc68cdb-6cf4-41ef-bf35-71bed2e7f5cf admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:10.888139 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc68cdb-6cf4-41ef-bf35-71bed2e7f5cf admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:10.958451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b7815-31fd-4be8-9bf9-99bd0f88473a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:10.959463 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b7815-31fd-4be8-9bf9-99bd0f88473a admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:10.960242 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b7815-31fd-4be8-9bf9-99bd0f88473a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:10.960507 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b7815-31fd-4be8-9bf9-99bd0f88473a admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:10.961214 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b7815-31fd-4be8-9bf9-99bd0f88473a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:10.963216 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b7815-31fd-4be8-9bf9-99bd0f88473a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:11.058679 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:11.068597 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1108/2200] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:37:10 2022] POST /identity/v3/users => generated 287 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:11.069894 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9db502-74ca-4ba2-b45c-c373f2cc19e5 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:11.081607 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9db502-74ca-4ba2-b45c-c373f2cc19e5 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:11.087336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a7b18-19d1-4df4-b26c-c23e4a0cbddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:11.091296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a7b18-19d1-4df4-b26c-c23e4a0cbddb admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:11.091296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a7b18-19d1-4df4-b26c-c23e4a0cbddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:11.091296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34a7b18-19d1-4df4-b26c-c23e4a0cbddb admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:11.091296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a7b18-19d1-4df4-b26c-c23e4a0cbddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:11.091296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34a7b18-19d1-4df4-b26c-c23e4a0cbddb admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:11.103422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1093/2201] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:11.106868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b7815-31fd-4be8-9bf9-99bd0f88473a admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:11.122768 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b7815-31fd-4be8-9bf9-99bd0f88473a admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:11.123230 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a7b18-19d1-4df4-b26c-c23e4a0cbddb admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:11.124999 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c34a7b18-19d1-4df4-b26c-c23e4a0cbddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:11.129108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab37de-0e25-424f-bac2-3f398cf88c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:11.130439 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab37de-0e25-424f-bac2-3f398cf88c8a admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:11.130763 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab37de-0e25-424f-bac2-3f398cf88c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:11.131152 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab37de-0e25-424f-bac2-3f398cf88c8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:11.132105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab37de-0e25-424f-bac2-3f398cf88c8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:11.133734 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab37de-0e25-424f-bac2-3f398cf88c8a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:11.142596 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34a7b18-19d1-4df4-b26c-c23e4a0cbddb admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:11.150423 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b51c3a7-6c68-4f73-97ee-e41fd3f3cb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:11.150423 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b51c3a7-6c68-4f73-97ee-e41fd3f3cb65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:11.150423 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b51c3a7-6c68-4f73-97ee-e41fd3f3cb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:11.151000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b51c3a7-6c68-4f73-97ee-e41fd3f3cb65 admin admin] PATH_INFO: `/v3/projects/d30f186e29664bd482f3e17b5dc2f4fe` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:11.151680 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1109/2202] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:11.152490 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab37de-0e25-424f-bac2-3f398cf88c8a admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:11.158069 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b51c3a7-6c68-4f73-97ee-e41fd3f3cb65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d30f186e29664bd482f3e17b5dc2f4fe)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:11.165205 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab37de-0e25-424f-bac2-3f398cf88c8a admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:11.167186 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b51c3a7-6c68-4f73-97ee-e41fd3f3cb65 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:11.182984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51614ff-aa55-4ddd-b31a-4bb2ae08c14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:11.182984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51614ff-aa55-4ddd-b31a-4bb2ae08c14c admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:11.182984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51614ff-aa55-4ddd-b31a-4bb2ae08c14c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:11.182984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51614ff-aa55-4ddd-b31a-4bb2ae08c14c admin admin] PATH_INFO: `/v3/projects/09b89ed90acc4dd2aa3bca17bb1a1fe9/users/1d76f23d7b614fac857a9aefd2d345b6/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:11.209996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51614ff-aa55-4ddd-b31a-4bb2ae08c14c admin admin] RBAC: Authorizing `identity:create_grant(project_id=09b89ed90acc4dd2aa3bca17bb1a1fe9, user_id=1d76f23d7b614fac857a9aefd2d345b6, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:11.213879 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51614ff-aa55-4ddd-b31a-4bb2ae08c14c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:11.306265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1110/2203] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:37:11 2022] PUT /identity/v3/projects/09b89ed90acc4dd2aa3bca17bb1a1fe9/users/1d76f23d7b614fac857a9aefd2d345b6/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:11.310526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67999f6-b320-4c7c-9ef2-e035885df831 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:11.310526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67999f6-b320-4c7c-9ef2-e035885df831 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:11.310526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67999f6-b320-4c7c-9ef2-e035885df831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:11.328898 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-d67999f6-b320-4c7c-9ef2-e035885df831 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:11.330127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d67999f6-b320-4c7c-9ef2-e035885df831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:11.393788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1111/2204] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:37:11.408364 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67999f6-b320-4c7c-9ef2-e035885df831 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:11.410322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d67999f6-b320-4c7c-9ef2-e035885df831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:11.427622 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0b51c3a7-6c68-4f73-97ee-e41fd3f3cb65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd30f186e29664bd482f3e17b5dc2f4fe', 'request_id': 'req-0b51c3a7-6c68-4f73-97ee-e41fd3f3cb65'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:11.521908 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0b51c3a7-6c68-4f73-97ee-e41fd3f3cb65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd30f186e29664bd482f3e17b5dc2f4fe', 'request_id': 'req-0b51c3a7-6c68-4f73-97ee-e41fd3f3cb65'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:11.539586 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0b51c3a7-6c68-4f73-97ee-e41fd3f3cb65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd30f186e29664bd482f3e17b5dc2f4fe', 'request_id': 'req-0b51c3a7-6c68-4f73-97ee-e41fd3f3cb65'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:11.566372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0b51c3a7-6c68-4f73-97ee-e41fd3f3cb65 admin admin] The token cache is being invalidate because project d30f186e29664bd482f3e17b5dc2f4fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:37:11.566372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0b51c3a7-6c68-4f73-97ee-e41fd3f3cb65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:11.571361 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0b51c3a7-6c68-4f73-97ee-e41fd3f3cb65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:11.574306 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67999f6-b320-4c7c-9ef2-e035885df831 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:11.576960 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1094/2205] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:37:11 2022] DELETE /identity/v3/projects/d30f186e29664bd482f3e17b5dc2f4fe => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:37:11.579056 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca0d89-06e4-4f4c-b3b1-50d12f603716 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:11.579347 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca0d89-06e4-4f4c-b3b1-50d12f603716 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:11.579911 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca0d89-06e4-4f4c-b3b1-50d12f603716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:11.605223 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-bdca0d89-06e4-4f4c-b3b1-50d12f603716 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:11.607254 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-bdca0d89-06e4-4f4c-b3b1-50d12f603716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:11.701989 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ec38a-6be5-4834-8be5-1d4b3eeb33de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:11.703930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ec38a-6be5-4834-8be5-1d4b3eeb33de service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:11.704397 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ec38a-6be5-4834-8be5-1d4b3eeb33de service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:11.704853 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ec38a-6be5-4834-8be5-1d4b3eeb33de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:11.707649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e26ec38a-6be5-4834-8be5-1d4b3eeb33de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:11.764978 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1095/2206] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:11.775117 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964db02-6881-44ff-88e9-73162aaca8b4 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:11.775117 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964db02-6881-44ff-88e9-73162aaca8b4 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:11.775117 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b964db02-6881-44ff-88e9-73162aaca8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:11.797049 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-b964db02-6881-44ff-88e9-73162aaca8b4 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:11.803385 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b964db02-6881-44ff-88e9-73162aaca8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:11.809148 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1096/2207] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:37:11.812959 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964db02-6881-44ff-88e9-73162aaca8b4 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:11.832584 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b964db02-6881-44ff-88e9-73162aaca8b4 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:11.838449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057f0ce-d6b9-49d6-9db9-8a9700b970e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:11.839165 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057f0ce-d6b9-49d6-9db9-8a9700b970e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:11.839450 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057f0ce-d6b9-49d6-9db9-8a9700b970e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:11.839730 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057f0ce-d6b9-49d6-9db9-8a9700b970e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:11.840469 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057f0ce-d6b9-49d6-9db9-8a9700b970e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:37:11.840781 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057f0ce-d6b9-49d6-9db9-8a9700b970e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:11.842384 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057f0ce-d6b9-49d6-9db9-8a9700b970e4 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:11.845574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ec38a-6be5-4834-8be5-1d4b3eeb33de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:11.847201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ec38a-6be5-4834-8be5-1d4b3eeb33de service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:11.858909 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1097/2208] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:37:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:37:11.859128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1112/2209] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:11.861505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057f0ce-d6b9-49d6-9db9-8a9700b970e4 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:11.872366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057f0ce-d6b9-49d6-9db9-8a9700b970e4 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:11.877985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a018a75-f96a-4a86-8657-5a3bf5e56cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:11.883136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a018a75-f96a-4a86-8657-5a3bf5e56cbf admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:11.883136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a018a75-f96a-4a86-8657-5a3bf5e56cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:11.883136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a018a75-f96a-4a86-8657-5a3bf5e56cbf admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:11.883136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a018a75-f96a-4a86-8657-5a3bf5e56cbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:11.883136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a018a75-f96a-4a86-8657-5a3bf5e56cbf admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:11.984063 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1098/2210] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:37:11 2022] POST /identity/v3/projects => generated 391 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:11.989161 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ec38a-6be5-4834-8be5-1d4b3eeb33de service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:12.002589 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ec38a-6be5-4834-8be5-1d4b3eeb33de service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:12.080022 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d35da-9d93-46d3-8eb7-577f5403e81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:12.080022 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d35da-9d93-46d3-8eb7-577f5403e81b admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:12.080022 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d35da-9d93-46d3-8eb7-577f5403e81b admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:12.080022 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d35da-9d93-46d3-8eb7-577f5403e81b admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:12.080507 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d35da-9d93-46d3-8eb7-577f5403e81b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:12.083076 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d35da-9d93-46d3-8eb7-577f5403e81b admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:12.169043 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1113/2211] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:37:11 2022] POST /identity/v3/users => generated 296 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:12.177414 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a018a75-f96a-4a86-8657-5a3bf5e56cbf admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:12.183500 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a018a75-f96a-4a86-8657-5a3bf5e56cbf admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:12.188238 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59837228-2d96-41f9-868d-8cdea125604f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:12.188894 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59837228-2d96-41f9-868d-8cdea125604f admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:12.189174 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59837228-2d96-41f9-868d-8cdea125604f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:12.189447 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59837228-2d96-41f9-868d-8cdea125604f admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:12.190169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59837228-2d96-41f9-868d-8cdea125604f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:12.191798 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59837228-2d96-41f9-868d-8cdea125604f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:12.198648 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:12.204629 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1099/2212] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:37:12.209543 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d35da-9d93-46d3-8eb7-577f5403e81b admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:12.228666 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d35da-9d93-46d3-8eb7-577f5403e81b admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:12.228666 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e74bb-65ab-4f10-8db9-50e862f20fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:12.228666 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e74bb-65ab-4f10-8db9-50e862f20fff admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:12.228666 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e74bb-65ab-4f10-8db9-50e862f20fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:12.229134 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e74bb-65ab-4f10-8db9-50e862f20fff admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:12.229134 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5e74bb-65ab-4f10-8db9-50e862f20fff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:12.230062 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5e74bb-65ab-4f10-8db9-50e862f20fff admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:12.246429 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1114/2213] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:37:12.250566 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59837228-2d96-41f9-868d-8cdea125604f admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:12.261533 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59837228-2d96-41f9-868d-8cdea125604f admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:12.266436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f356fce-c752-432d-ae1d-9319a0257845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:12.267381 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f356fce-c752-432d-ae1d-9319a0257845 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:12.267653 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f356fce-c752-432d-ae1d-9319a0257845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:12.267923 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f356fce-c752-432d-ae1d-9319a0257845 admin admin] PATH_INFO: `/v3/projects/1b85c126682b4cfbaf6963c88cae9dfd/users/70cf41938b14463096a18db37ec16400/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:12.290351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f356fce-c752-432d-ae1d-9319a0257845 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b85c126682b4cfbaf6963c88cae9dfd, user_id=70cf41938b14463096a18db37ec16400, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:12.293514 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f356fce-c752-432d-ae1d-9319a0257845 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:12.299459 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e74bb-65ab-4f10-8db9-50e862f20fff admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:12.301787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ce5e74bb-65ab-4f10-8db9-50e862f20fff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:12.315706 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e74bb-65ab-4f10-8db9-50e862f20fff admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:12.320601 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63c51d1-05a9-4235-90fa-0b82c3cd51ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:12.321465 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c51d1-05a9-4235-90fa-0b82c3cd51ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:12.321751 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c51d1-05a9-4235-90fa-0b82c3cd51ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:12.322022 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63c51d1-05a9-4235-90fa-0b82c3cd51ee admin admin] PATH_INFO: `/v3/users/58205a78eae64db4a5f98a0726813728` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:12.323282 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63c51d1-05a9-4235-90fa-0b82c3cd51ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=58205a78eae64db4a5f98a0726813728)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:12.325181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63c51d1-05a9-4235-90fa-0b82c3cd51ee admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:12.406478 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1100/2214] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:37:12 2022] PUT /identity/v3/projects/1b85c126682b4cfbaf6963c88cae9dfd/users/70cf41938b14463096a18db37ec16400/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:37:12.416765 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489e8ea-babc-470f-a502-6d84831bcdcb None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:12.417062 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489e8ea-babc-470f-a502-6d84831bcdcb None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:12.417338 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3489e8ea-babc-470f-a502-6d84831bcdcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:12.452612 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-3489e8ea-babc-470f-a502-6d84831bcdcb None None] MFA Rules not processed for user `70cf41938b14463096a18db37ec16400`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:12.453875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3489e8ea-babc-470f-a502-6d84831bcdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:12.545346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e63c51d1-05a9-4235-90fa-0b82c3cd51ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58205a78eae64db4a5f98a0726813728'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:12.566648 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1101/2215] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:37:12.571424 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489e8ea-babc-470f-a502-6d84831bcdcb None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:12.638419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e63c51d1-05a9-4235-90fa-0b82c3cd51ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58205a78eae64db4a5f98a0726813728'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:12.654945 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e63c51d1-05a9-4235-90fa-0b82c3cd51ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58205a78eae64db4a5f98a0726813728'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:12.655970 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e63c51d1-05a9-4235-90fa-0b82c3cd51ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58205a78eae64db4a5f98a0726813728'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:12.656892 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3489e8ea-babc-470f-a502-6d84831bcdcb None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:12.668166 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48fa38-14a7-489f-869a-410ecdd5df3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:12.669060 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48fa38-14a7-489f-869a-410ecdd5df3d admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:12.669504 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48fa38-14a7-489f-869a-410ecdd5df3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:12.670618 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48fa38-14a7-489f-869a-410ecdd5df3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:12.672170 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac48fa38-14a7-489f-869a-410ecdd5df3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:12.677089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac48fa38-14a7-489f-869a-410ecdd5df3d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:12.677211 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e63c51d1-05a9-4235-90fa-0b82c3cd51ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58205a78eae64db4a5f98a0726813728'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:12.690957 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1115/2216] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:37:12 2022] DELETE /identity/v3/users/58205a78eae64db4a5f98a0726813728 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:37:12.693798 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85569d-6392-41a2-ab08-79b0f96915f0 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:12.694346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85569d-6392-41a2-ab08-79b0f96915f0 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:12.694865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85569d-6392-41a2-ab08-79b0f96915f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:12.721357 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-3e85569d-6392-41a2-ab08-79b0f96915f0 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:12.722866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3e85569d-6392-41a2-ab08-79b0f96915f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:12.840857 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1116/2217] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:12.840969 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1102/2218] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:37:12 2022] POST /identity/v3/projects => generated 391 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:12.842551 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e85569d-6392-41a2-ab08-79b0f96915f0 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:12.852886 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3e85569d-6392-41a2-ab08-79b0f96915f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:12.859558 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48fa38-14a7-489f-869a-410ecdd5df3d admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:12.861399 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ac48fa38-14a7-489f-869a-410ecdd5df3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:12.887151 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e85569d-6392-41a2-ab08-79b0f96915f0 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:12.977203 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48fa38-14a7-489f-869a-410ecdd5df3d admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:12.987436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606cfb7b-dd46-4731-8125-88f959b1c3d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:12.989577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606cfb7b-dd46-4731-8125-88f959b1c3d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:12.989862 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606cfb7b-dd46-4731-8125-88f959b1c3d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:12.991239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606cfb7b-dd46-4731-8125-88f959b1c3d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:12.999256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40a858-09c8-4f96-87a6-9cd5863e3c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:12.999256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40a858-09c8-4f96-87a6-9cd5863e3c3d admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:12.999256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40a858-09c8-4f96-87a6-9cd5863e3c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:12.999256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40a858-09c8-4f96-87a6-9cd5863e3c3d admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:12.999624 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40a858-09c8-4f96-87a6-9cd5863e3c3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:13.000409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40a858-09c8-4f96-87a6-9cd5863e3c3d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:13.022582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606cfb7b-dd46-4731-8125-88f959b1c3d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:13.024129 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606cfb7b-dd46-4731-8125-88f959b1c3d0 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:13.038240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1103/2219] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:37:13.102956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1117/2220] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:37:12 2022] POST /identity/v3/users => generated 302 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:13.107080 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606cfb7b-dd46-4731-8125-88f959b1c3d0 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:13.118952 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606cfb7b-dd46-4731-8125-88f959b1c3d0 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:13.124146 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa2f5c-3bcd-4fbe-9c45-b312777b537f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:13.125050 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa2f5c-3bcd-4fbe-9c45-b312777b537f admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:13.125410 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa2f5c-3bcd-4fbe-9c45-b312777b537f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:13.125759 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faa2f5c-3bcd-4fbe-9c45-b312777b537f admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:13.127509 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa2f5c-3bcd-4fbe-9c45-b312777b537f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:13.129091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faa2f5c-3bcd-4fbe-9c45-b312777b537f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:13.143175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1104/2221] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:13.148779 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40a858-09c8-4f96-87a6-9cd5863e3c3d admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:13.159095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40a858-09c8-4f96-87a6-9cd5863e3c3d admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:13.165639 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f49a6-7d6e-4912-84b9-03ad1603c11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:13.166106 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f49a6-7d6e-4912-84b9-03ad1603c11c admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:13.166385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f49a6-7d6e-4912-84b9-03ad1603c11c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:13.166653 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2f49a6-7d6e-4912-84b9-03ad1603c11c admin admin] PATH_INFO: `/v3/projects/3f80d9e7b9964069a5b143e40135a665/users/8ba205d768b94a90b0a1d26225501d97/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:13.184840 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2f49a6-7d6e-4912-84b9-03ad1603c11c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f80d9e7b9964069a5b143e40135a665, user_id=8ba205d768b94a90b0a1d26225501d97, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:13.187608 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2f49a6-7d6e-4912-84b9-03ad1603c11c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:13.382861 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1118/2222] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:37:13 2022] PUT /identity/v3/projects/3f80d9e7b9964069a5b143e40135a665/users/8ba205d768b94a90b0a1d26225501d97/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:13.387014 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa2f5c-3bcd-4fbe-9c45-b312777b537f admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:13.398241 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faa2f5c-3bcd-4fbe-9c45-b312777b537f admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:13.463358 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d9c80d-043c-4d8d-a68b-cfc508e96f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:13.464137 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9c80d-043c-4d8d-a68b-cfc508e96f4e admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:13.464478 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9c80d-043c-4d8d-a68b-cfc508e96f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:13.464761 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d9c80d-043c-4d8d-a68b-cfc508e96f4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:13.465495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d9c80d-043c-4d8d-a68b-cfc508e96f4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:13.467138 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d9c80d-043c-4d8d-a68b-cfc508e96f4e admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:13.480707 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1105/2223] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:37:13.482748 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f49a6-7d6e-4912-84b9-03ad1603c11c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:13.493496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2f49a6-7d6e-4912-84b9-03ad1603c11c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:13.503627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55847b02-1228-4ed9-bbd1-5ca25bfa5ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:13.504541 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55847b02-1228-4ed9-bbd1-5ca25bfa5ca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:13.504828 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55847b02-1228-4ed9-bbd1-5ca25bfa5ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:13.505109 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55847b02-1228-4ed9-bbd1-5ca25bfa5ca2 admin admin] PATH_INFO: `/v3/projects/3f80d9e7b9964069a5b143e40135a665/users/8ba205d768b94a90b0a1d26225501d97/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:13.507648 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55847b02-1228-4ed9-bbd1-5ca25bfa5ca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f80d9e7b9964069a5b143e40135a665, user_id=8ba205d768b94a90b0a1d26225501d97, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:13.510974 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55847b02-1228-4ed9-bbd1-5ca25bfa5ca2 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:13.525291 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1119/2224] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:37:13 2022] PUT /identity/v3/projects/3f80d9e7b9964069a5b143e40135a665/users/8ba205d768b94a90b0a1d26225501d97/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:13.536582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96774d9-1ccb-4f74-993e-c91138659cbc None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:13.536907 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96774d9-1ccb-4f74-993e-c91138659cbc None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:13.537188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96774d9-1ccb-4f74-993e-c91138659cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:13.572951 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-f96774d9-1ccb-4f74-993e-c91138659cbc None None] MFA Rules not processed for user `8ba205d768b94a90b0a1d26225501d97`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:13.575187 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f96774d9-1ccb-4f74-993e-c91138659cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:13.658956 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1106/2225] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:13 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:13.670094 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55847b02-1228-4ed9-bbd1-5ca25bfa5ca2 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:13.671924 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-55847b02-1228-4ed9-bbd1-5ca25bfa5ca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:13.731921 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55847b02-1228-4ed9-bbd1-5ca25bfa5ca2 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:13.735679 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d277a-f1bc-42e6-9adf-9ad88e4296fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:13.736353 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d277a-f1bc-42e6-9adf-9ad88e4296fb service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:13.736636 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d277a-f1bc-42e6-9adf-9ad88e4296fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:13.736912 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d277a-f1bc-42e6-9adf-9ad88e4296fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:13.745819 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d277a-f1bc-42e6-9adf-9ad88e4296fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:13.747948 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d277a-f1bc-42e6-9adf-9ad88e4296fb service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:13.758944 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1120/2226] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:13 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:13.807643 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96774d9-1ccb-4f74-993e-c91138659cbc None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:13.810666 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d277a-f1bc-42e6-9adf-9ad88e4296fb service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:13.812284 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f96774d9-1ccb-4f74-993e-c91138659cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:13.812623 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8a6d277a-f1bc-42e6-9adf-9ad88e4296fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:13.825647 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d277a-f1bc-42e6-9adf-9ad88e4296fb service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:13.832658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9827a250-32a8-4a60-88d8-7b69df4f00d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:13.833322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827a250-32a8-4a60-88d8-7b69df4f00d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:13.833616 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827a250-32a8-4a60-88d8-7b69df4f00d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:13.834418 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827a250-32a8-4a60-88d8-7b69df4f00d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:13.845775 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-9827a250-32a8-4a60-88d8-7b69df4f00d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:13.906453 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96774d9-1ccb-4f74-993e-c91138659cbc None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:13.912714 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c499bc-54f0-444f-b80c-ad77352a4059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:13.913726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c499bc-54f0-444f-b80c-ad77352a4059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:13.914300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c499bc-54f0-444f-b80c-ad77352a4059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:13.914666 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c499bc-54f0-444f-b80c-ad77352a4059 admin admin] PATH_INFO: `/v3/projects/62f595fe610a403993ae86f3899eb494` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:13.916136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c499bc-54f0-444f-b80c-ad77352a4059 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62f595fe610a403993ae86f3899eb494)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:13.918299 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c499bc-54f0-444f-b80c-ad77352a4059 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:13.946416 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9827a250-32a8-4a60-88d8-7b69df4f00d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:13.948207 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9827a250-32a8-4a60-88d8-7b69df4f00d9 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:13.969947 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1121/2227] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:13.977640 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada3c61-20c4-4c7c-a43c-f019dcc8b8f6 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:13.977640 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada3c61-20c4-4c7c-a43c-f019dcc8b8f6 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:13.977867 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada3c61-20c4-4c7c-a43c-f019dcc8b8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:14.025386 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-dada3c61-20c4-4c7c-a43c-f019dcc8b8f6 None None] MFA Rules not processed for user `1d76f23d7b614fac857a9aefd2d345b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:14.028515 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-dada3c61-20c4-4c7c-a43c-f019dcc8b8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:14.097051 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:14.130043 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-15c499bc-54f0-444f-b80c-ad77352a4059 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62f595fe610a403993ae86f3899eb494', 'request_id': 'req-15c499bc-54f0-444f-b80c-ad77352a4059'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:14.149315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1122/2228] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:37:14.175761 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada3c61-20c4-4c7c-a43c-f019dcc8b8f6 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:14.177364 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-dada3c61-20c4-4c7c-a43c-f019dcc8b8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:14.191835 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada3c61-20c4-4c7c-a43c-f019dcc8b8f6 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:14.231133 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-15c499bc-54f0-444f-b80c-ad77352a4059 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62f595fe610a403993ae86f3899eb494', 'request_id': 'req-15c499bc-54f0-444f-b80c-ad77352a4059'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:14.239612 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-15c499bc-54f0-444f-b80c-ad77352a4059 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62f595fe610a403993ae86f3899eb494', 'request_id': 'req-15c499bc-54f0-444f-b80c-ad77352a4059'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:14.252819 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-15c499bc-54f0-444f-b80c-ad77352a4059 admin admin] The token cache is being invalidate because project 62f595fe610a403993ae86f3899eb494 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:37:14.253108 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-15c499bc-54f0-444f-b80c-ad77352a4059 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:14.253761 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-15c499bc-54f0-444f-b80c-ad77352a4059 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:14.255353 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1107/2229] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:37:13 2022] DELETE /identity/v3/projects/62f595fe610a403993ae86f3899eb494 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:14.256735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c499bc-54f0-444f-b80c-ad77352a4059 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:14.258136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-15c499bc-54f0-444f-b80c-ad77352a4059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:14.261390 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ebc870-a91e-4391-ab74-a32edbe8d6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:14.263367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ebc870-a91e-4391-ab74-a32edbe8d6eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:14.263367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ebc870-a91e-4391-ab74-a32edbe8d6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:14.263367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ebc870-a91e-4391-ab74-a32edbe8d6eb admin admin] PATH_INFO: `/v3/users/d03330e6199244449437db7d3d99ae8f` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:14.264622 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ebc870-a91e-4391-ab74-a32edbe8d6eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d03330e6199244449437db7d3d99ae8f)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:14.267102 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ebc870-a91e-4391-ab74-a32edbe8d6eb admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:14.327861 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c499bc-54f0-444f-b80c-ad77352a4059 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:14.331534 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db48e0-f1e1-4a66-8f71-1c5f1e957912 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:14.332319 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db48e0-f1e1-4a66-8f71-1c5f1e957912 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:14.332599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db48e0-f1e1-4a66-8f71-1c5f1e957912 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:14.332869 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db48e0-f1e1-4a66-8f71-1c5f1e957912 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:14.335324 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-23db48e0-f1e1-4a66-8f71-1c5f1e957912 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:14.400538 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db48e0-f1e1-4a66-8f71-1c5f1e957912 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:14.401937 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23db48e0-f1e1-4a66-8f71-1c5f1e957912 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:14.411678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1108/2230] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:14.412867 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db48e0-f1e1-4a66-8f71-1c5f1e957912 service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:14.414152 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-23db48e0-f1e1-4a66-8f71-1c5f1e957912 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:14.430538 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db48e0-f1e1-4a66-8f71-1c5f1e957912 service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:14.435960 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae7658-a1ee-4dfd-bb0b-3dfba852afd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:14.436922 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae7658-a1ee-4dfd-bb0b-3dfba852afd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:14.437332 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae7658-a1ee-4dfd-bb0b-3dfba852afd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:14.437834 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae7658-a1ee-4dfd-bb0b-3dfba852afd7 admin admin] PATH_INFO: `/v3/users/eebc19aad8b84a7396835091a6e00a64` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:14.442547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae7658-a1ee-4dfd-bb0b-3dfba852afd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eebc19aad8b84a7396835091a6e00a64)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:14.445381 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae7658-a1ee-4dfd-bb0b-3dfba852afd7 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:14.488911 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-46ebc870-a91e-4391-ab74-a32edbe8d6eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd03330e6199244449437db7d3d99ae8f'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:14.575931 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-46ebc870-a91e-4391-ab74-a32edbe8d6eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd03330e6199244449437db7d3d99ae8f'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:14.601335 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-46ebc870-a91e-4391-ab74-a32edbe8d6eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd03330e6199244449437db7d3d99ae8f'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:14.602075 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-46ebc870-a91e-4391-ab74-a32edbe8d6eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd03330e6199244449437db7d3d99ae8f'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:14.609953 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-46ebc870-a91e-4391-ab74-a32edbe8d6eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd03330e6199244449437db7d3d99ae8f'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:14.618640 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1123/2231] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:37:14 2022] DELETE /identity/v3/users/d03330e6199244449437db7d3d99ae8f => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:37:14.622174 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584f07a-7702-4be1-a681-6eea9bed7d1c None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:14.622464 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584f07a-7702-4be1-a681-6eea9bed7d1c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:14.622741 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584f07a-7702-4be1-a681-6eea9bed7d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:14.640669 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-7584f07a-7702-4be1-a681-6eea9bed7d1c None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:14.641863 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-7584f07a-7702-4be1-a681-6eea9bed7d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:14.676368 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-67ae7658-a1ee-4dfd-bb0b-3dfba852afd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eebc19aad8b84a7396835091a6e00a64'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:14.713784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1124/2232] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:37:14.725345 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584f07a-7702-4be1-a681-6eea9bed7d1c None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:14.726921 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-7584f07a-7702-4be1-a681-6eea9bed7d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:14.730676 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-67ae7658-a1ee-4dfd-bb0b-3dfba852afd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eebc19aad8b84a7396835091a6e00a64'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:14.731819 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-67ae7658-a1ee-4dfd-bb0b-3dfba852afd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eebc19aad8b84a7396835091a6e00a64'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:14.740002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-67ae7658-a1ee-4dfd-bb0b-3dfba852afd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eebc19aad8b84a7396835091a6e00a64'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:14.740724 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-67ae7658-a1ee-4dfd-bb0b-3dfba852afd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eebc19aad8b84a7396835091a6e00a64'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:14.782870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1109/2233] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:37:14 2022] DELETE /identity/v3/users/eebc19aad8b84a7396835091a6e00a64 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:14.826037 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584f07a-7702-4be1-a681-6eea9bed7d1c None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:14.903627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b655a-e8a3-4994-bf98-6822c9be2e9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:14.904350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b655a-e8a3-4994-bf98-6822c9be2e9f service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:14.904639 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b655a-e8a3-4994-bf98-6822c9be2e9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:14.904917 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b655a-e8a3-4994-bf98-6822c9be2e9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:14.907847 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a31b655a-e8a3-4994-bf98-6822c9be2e9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:14.989645 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b655a-e8a3-4994-bf98-6822c9be2e9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:14.991860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b655a-e8a3-4994-bf98-6822c9be2e9f service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:15.005000 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1125/2234] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:37:15.404386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae7658-a1ee-4dfd-bb0b-3dfba852afd7 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:15.405837 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-67ae7658-a1ee-4dfd-bb0b-3dfba852afd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:15.448232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae7658-a1ee-4dfd-bb0b-3dfba852afd7 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:15.448232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c175f30-ca8c-4fd1-b7ac-af83852c51b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:15.454142 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c175f30-ca8c-4fd1-b7ac-af83852c51b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:15.454142 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c175f30-ca8c-4fd1-b7ac-af83852c51b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:15.454142 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c175f30-ca8c-4fd1-b7ac-af83852c51b1 admin admin] PATH_INFO: `/v3/projects/7156a444d77b40d5b56b6b62122ce14e` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:15.454142 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c175f30-ca8c-4fd1-b7ac-af83852c51b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7156a444d77b40d5b56b6b62122ce14e)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:15.454142 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c175f30-ca8c-4fd1-b7ac-af83852c51b1 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:15.597713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0c175f30-ca8c-4fd1-b7ac-af83852c51b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7156a444d77b40d5b56b6b62122ce14e', 'request_id': 'req-0c175f30-ca8c-4fd1-b7ac-af83852c51b1'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:15.695154 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0c175f30-ca8c-4fd1-b7ac-af83852c51b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7156a444d77b40d5b56b6b62122ce14e', 'request_id': 'req-0c175f30-ca8c-4fd1-b7ac-af83852c51b1'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:15.703285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0c175f30-ca8c-4fd1-b7ac-af83852c51b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7156a444d77b40d5b56b6b62122ce14e', 'request_id': 'req-0c175f30-ca8c-4fd1-b7ac-af83852c51b1'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:15.704199 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b655a-e8a3-4994-bf98-6822c9be2e9f service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:15.710180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a31b655a-e8a3-4994-bf98-6822c9be2e9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:15.717019 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0c175f30-ca8c-4fd1-b7ac-af83852c51b1 admin admin] The token cache is being invalidate because project 7156a444d77b40d5b56b6b62122ce14e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:37:15.717308 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0c175f30-ca8c-4fd1-b7ac-af83852c51b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:15.718130 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0c175f30-ca8c-4fd1-b7ac-af83852c51b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:15.723335 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1110/2235] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:37:15 2022] DELETE /identity/v3/projects/7156a444d77b40d5b56b6b62122ce14e => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:15.734795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c797a5-f5ea-4cad-822c-60be4c50ef1e None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:15.736342 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b655a-e8a3-4994-bf98-6822c9be2e9f service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:15.736790 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c797a5-f5ea-4cad-822c-60be4c50ef1e None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:15.736790 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c797a5-f5ea-4cad-822c-60be4c50ef1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:15.757644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-f4c797a5-f5ea-4cad-822c-60be4c50ef1e None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:15.759724 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f4c797a5-f5ea-4cad-822c-60be4c50ef1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:15.803959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b1aec-6855-4647-aefe-833d7b0c6356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:15.805111 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b1aec-6855-4647-aefe-833d7b0c6356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:15.805792 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b1aec-6855-4647-aefe-833d7b0c6356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:15.806288 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b1aec-6855-4647-aefe-833d7b0c6356 admin admin] PATH_INFO: `/v3/projects/20fa1221b15f438baf91f6b6293e27c8` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:15.807525 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b1aec-6855-4647-aefe-833d7b0c6356 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20fa1221b15f438baf91f6b6293e27c8)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:15.810931 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b1aec-6855-4647-aefe-833d7b0c6356 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:15.822107 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1111/2236] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:15.830278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4d837-f87d-4f23-b01f-545c9ff36491 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:15.830574 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4d837-f87d-4f23-b01f-545c9ff36491 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:15.830907 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4d837-f87d-4f23-b01f-545c9ff36491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:15.853998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-bfc4d837-f87d-4f23-b01f-545c9ff36491 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:15.855150 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-bfc4d837-f87d-4f23-b01f-545c9ff36491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:15.866869 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1112/2237] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:37:15.866869 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:15.870330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4d837-f87d-4f23-b01f-545c9ff36491 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:15.879673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4d837-f87d-4f23-b01f-545c9ff36491 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:15.885188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a522f-8c1a-4e04-ac14-64951c2be426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:15.885853 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a522f-8c1a-4e04-ac14-64951c2be426 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:15.886134 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a522f-8c1a-4e04-ac14-64951c2be426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:15.886411 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a522f-8c1a-4e04-ac14-64951c2be426 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:15.887182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a522f-8c1a-4e04-ac14-64951c2be426 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:37:15.887490 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a522f-8c1a-4e04-ac14-64951c2be426 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:15.889124 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a522f-8c1a-4e04-ac14-64951c2be426 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:15.902909 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1113/2238] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:37:15.905837 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a522f-8c1a-4e04-ac14-64951c2be426 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:15.915902 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a522f-8c1a-4e04-ac14-64951c2be426 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:15.921109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd8c53-560d-42fb-a9b6-ef44cec525c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:15.921685 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd8c53-560d-42fb-a9b6-ef44cec525c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:15.922407 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd8c53-560d-42fb-a9b6-ef44cec525c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:15.922407 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd8c53-560d-42fb-a9b6-ef44cec525c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:15.922948 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd8c53-560d-42fb-a9b6-ef44cec525c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:15.925003 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd8c53-560d-42fb-a9b6-ef44cec525c4 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:15.942676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b85b1aec-6855-4647-aefe-833d7b0c6356 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20fa1221b15f438baf91f6b6293e27c8', 'request_id': 'req-b85b1aec-6855-4647-aefe-833d7b0c6356'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:16.017137 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1114/2239] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:37:15 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:37:16.021824 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd8c53-560d-42fb-a9b6-ef44cec525c4 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:16.032461 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b85b1aec-6855-4647-aefe-833d7b0c6356 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20fa1221b15f438baf91f6b6293e27c8', 'request_id': 'req-b85b1aec-6855-4647-aefe-833d7b0c6356'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:16.039107 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b85b1aec-6855-4647-aefe-833d7b0c6356 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20fa1221b15f438baf91f6b6293e27c8', 'request_id': 'req-b85b1aec-6855-4647-aefe-833d7b0c6356'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:16.053787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b85b1aec-6855-4647-aefe-833d7b0c6356 admin admin] The token cache is being invalidate because project 20fa1221b15f438baf91f6b6293e27c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:37:16.054124 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b85b1aec-6855-4647-aefe-833d7b0c6356 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:16.054728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b85b1aec-6855-4647-aefe-833d7b0c6356 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:16.056418 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1126/2240] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:37:15 2022] DELETE /identity/v3/projects/20fa1221b15f438baf91f6b6293e27c8 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:16.063577 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2cccf-c307-4541-8e32-9a807e6a38b1 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:16.063859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2cccf-c307-4541-8e32-9a807e6a38b1 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:16.064131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c2cccf-c307-4541-8e32-9a807e6a38b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:16.088994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd8c53-560d-42fb-a9b6-ef44cec525c4 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:16.094629 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-13c2cccf-c307-4541-8e32-9a807e6a38b1 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:16.094826 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0dad0-9119-4de9-8518-9defb3ebb277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:16.094826 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0dad0-9119-4de9-8518-9defb3ebb277 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:16.095059 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-13c2cccf-c307-4541-8e32-9a807e6a38b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:16.095151 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0dad0-9119-4de9-8518-9defb3ebb277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:16.095151 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0dad0-9119-4de9-8518-9defb3ebb277 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:16.095966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff0dad0-9119-4de9-8518-9defb3ebb277 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:16.097900 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff0dad0-9119-4de9-8518-9defb3ebb277 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:16.105242 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1127/2241] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:16.113433 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef335de-450e-4842-b708-bee636e53ea2 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:16.113877 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef335de-450e-4842-b708-bee636e53ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:16.114154 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef335de-450e-4842-b708-bee636e53ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:16.135713 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-5ef335de-450e-4842-b708-bee636e53ea2 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:16.136927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5ef335de-450e-4842-b708-bee636e53ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:16.145291 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1128/2242] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:16.149747 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef335de-450e-4842-b708-bee636e53ea2 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:16.160558 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef335de-450e-4842-b708-bee636e53ea2 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:16.166536 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833ce3c-748a-41ef-92d9-6cf0ebdd7c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:16.167432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833ce3c-748a-41ef-92d9-6cf0ebdd7c1d admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:16.167728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833ce3c-748a-41ef-92d9-6cf0ebdd7c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:16.168014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833ce3c-748a-41ef-92d9-6cf0ebdd7c1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:16.168750 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833ce3c-748a-41ef-92d9-6cf0ebdd7c1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:37:16.169060 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833ce3c-748a-41ef-92d9-6cf0ebdd7c1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:16.170950 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833ce3c-748a-41ef-92d9-6cf0ebdd7c1d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:16.174666 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1115/2243] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:37:16 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:37:16.178189 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0dad0-9119-4de9-8518-9defb3ebb277 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:16.180790 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-9ff0dad0-9119-4de9-8518-9defb3ebb277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:16.190882 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1129/2244] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:37:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:37:16.194551 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833ce3c-748a-41ef-92d9-6cf0ebdd7c1d admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:16.196741 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0dad0-9119-4de9-8518-9defb3ebb277 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:16.201787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2bf3a-59d1-471b-8040-9f0ca6d0bd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:16.202799 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2bf3a-59d1-471b-8040-9f0ca6d0bd45 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:16.203196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2bf3a-59d1-471b-8040-9f0ca6d0bd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:16.203521 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2bf3a-59d1-471b-8040-9f0ca6d0bd45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:16.204460 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2bf3a-59d1-471b-8040-9f0ca6d0bd45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:16.205732 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833ce3c-748a-41ef-92d9-6cf0ebdd7c1d admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:16.207314 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2bf3a-59d1-471b-8040-9f0ca6d0bd45 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:16.211919 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b556f7-cb5f-4536-abc2-ba6d6683c04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:16.212818 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b556f7-cb5f-4536-abc2-ba6d6683c04e admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:16.213451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b556f7-cb5f-4536-abc2-ba6d6683c04e admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:16.213890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b556f7-cb5f-4536-abc2-ba6d6683c04e admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:16.214734 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b556f7-cb5f-4536-abc2-ba6d6683c04e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:16.216969 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b556f7-cb5f-4536-abc2-ba6d6683c04e admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:16.229118 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1116/2245] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:37:16.231880 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2bf3a-59d1-471b-8040-9f0ca6d0bd45 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:16.245371 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2bf3a-59d1-471b-8040-9f0ca6d0bd45 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:16.251433 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f02a18-5fd9-4a8a-a306-56ce3fbf789d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:16.252109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f02a18-5fd9-4a8a-a306-56ce3fbf789d admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:16.252402 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f02a18-5fd9-4a8a-a306-56ce3fbf789d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:16.252680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f02a18-5fd9-4a8a-a306-56ce3fbf789d admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:16.253413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f02a18-5fd9-4a8a-a306-56ce3fbf789d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:16.255067 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f02a18-5fd9-4a8a-a306-56ce3fbf789d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:16.283115 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1117/2246] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:16.284501 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f02a18-5fd9-4a8a-a306-56ce3fbf789d admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:16.298070 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f02a18-5fd9-4a8a-a306-56ce3fbf789d admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:16.306788 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31caa2-0bb7-4576-b80a-f079214fadd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:16.307737 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31caa2-0bb7-4576-b80a-f079214fadd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:16.308022 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31caa2-0bb7-4576-b80a-f079214fadd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:16.309411 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31caa2-0bb7-4576-b80a-f079214fadd3 admin admin] PATH_INFO: `/v3/projects/41295e76fc1c400db439f17d1cf64d95/users/7595af6e709e4538b3bb2d13a6461815/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:16.325667 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31caa2-0bb7-4576-b80a-f079214fadd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41295e76fc1c400db439f17d1cf64d95, user_id=7595af6e709e4538b3bb2d13a6461815, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:16.329108 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31caa2-0bb7-4576-b80a-f079214fadd3 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:16.347099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1130/2247] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:37:16 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:16.347099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973354d1-0dab-47d7-9bd2-d8a71743f6d5 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:16.347099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973354d1-0dab-47d7-9bd2-d8a71743f6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:16.347099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973354d1-0dab-47d7-9bd2-d8a71743f6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:16.366619 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-973354d1-0dab-47d7-9bd2-d8a71743f6d5 None None] MFA Rules not processed for user `1d76f23d7b614fac857a9aefd2d345b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:16.368712 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-973354d1-0dab-47d7-9bd2-d8a71743f6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:16.433692 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1131/2248] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:16.438952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973354d1-0dab-47d7-9bd2-d8a71743f6d5 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:16.468936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1118/2249] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:37:16 2022] PUT /identity/v3/projects/41295e76fc1c400db439f17d1cf64d95/users/7595af6e709e4538b3bb2d13a6461815/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:16.486388 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fd7e0-a449-4908-a932-fc41e76f47f3 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:16.487053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fd7e0-a449-4908-a932-fc41e76f47f3 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:16.487486 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81fd7e0-a449-4908-a932-fc41e76f47f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:16.551054 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973354d1-0dab-47d7-9bd2-d8a71743f6d5 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:16.551326 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e81fd7e0-a449-4908-a932-fc41e76f47f3 None None] MFA Rules not processed for user `7595af6e709e4538b3bb2d13a6461815`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:16.553193 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e81fd7e0-a449-4908-a932-fc41e76f47f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:16.649665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33db920-cabc-4d73-8f89-f158d10871e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:16.650955 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33db920-cabc-4d73-8f89-f158d10871e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:16.651375 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33db920-cabc-4d73-8f89-f158d10871e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:16.651806 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33db920-cabc-4d73-8f89-f158d10871e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:16.653627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33db920-cabc-4d73-8f89-f158d10871e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:16.656130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33db920-cabc-4d73-8f89-f158d10871e5 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:16.693003 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1119/2250] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:16.708977 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fd7e0-a449-4908-a932-fc41e76f47f3 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:16.709664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e81fd7e0-a449-4908-a932-fc41e76f47f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:16.785965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81fd7e0-a449-4908-a932-fc41e76f47f3 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:16.796081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d255f-4e2a-421f-a574-1fc995c29722 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:16.796081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d255f-4e2a-421f-a574-1fc995c29722 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:16.796081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d255f-4e2a-421f-a574-1fc995c29722 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:16.796081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d255f-4e2a-421f-a574-1fc995c29722 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:16.801596 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d255f-4e2a-421f-a574-1fc995c29722 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:16.804223 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d255f-4e2a-421f-a574-1fc995c29722 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:16.816070 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1120/2251] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:37:16.825986 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1132/2252] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:37:16 2022] POST /identity/v3/users => generated 288 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:16.848268 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d255f-4e2a-421f-a574-1fc995c29722 service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:16.848268 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d255f-4e2a-421f-a574-1fc995c29722 service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:16.856863 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f471e75f-6834-4f47-8b62-94c43d65fc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:16.857557 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471e75f-6834-4f47-8b62-94c43d65fc56 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:16.857904 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471e75f-6834-4f47-8b62-94c43d65fc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:16.858173 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f471e75f-6834-4f47-8b62-94c43d65fc56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:16.858934 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f471e75f-6834-4f47-8b62-94c43d65fc56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:16.862586 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f471e75f-6834-4f47-8b62-94c43d65fc56 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:16.888528 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1121/2253] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:16.890115 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33db920-cabc-4d73-8f89-f158d10871e5 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:16.903849 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33db920-cabc-4d73-8f89-f158d10871e5 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:16.912820 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d465579a-26fd-43ff-9aae-7c4c5abec4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:16.914763 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465579a-26fd-43ff-9aae-7c4c5abec4ab admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:16.915325 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465579a-26fd-43ff-9aae-7c4c5abec4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:16.916009 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465579a-26fd-43ff-9aae-7c4c5abec4ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:16.917473 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d465579a-26fd-43ff-9aae-7c4c5abec4ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:16.920483 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d465579a-26fd-43ff-9aae-7c4c5abec4ab admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:16.938170 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1133/2254] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:16 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:16.940883 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f471e75f-6834-4f47-8b62-94c43d65fc56 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:16.955131 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f471e75f-6834-4f47-8b62-94c43d65fc56 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:16.961024 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07fab98-8796-4f7d-94eb-508d83d42da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:16.961913 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fab98-8796-4f7d-94eb-508d83d42da6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:16.962199 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fab98-8796-4f7d-94eb-508d83d42da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:16.962891 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fab98-8796-4f7d-94eb-508d83d42da6 admin admin] PATH_INFO: `/v3/projects/7a486708a40b42ceb41394312e4da42e/users/fcba07a5d5a84bb4b1123d07192ad2e1/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:16.995675 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07fab98-8796-4f7d-94eb-508d83d42da6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a486708a40b42ceb41394312e4da42e, user_id=fcba07a5d5a84bb4b1123d07192ad2e1, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:16.998070 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07fab98-8796-4f7d-94eb-508d83d42da6 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:17.117069 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1122/2255] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:37:16 2022] PUT /identity/v3/projects/7a486708a40b42ceb41394312e4da42e/users/fcba07a5d5a84bb4b1123d07192ad2e1/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:17.136916 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af069dac-9071-444a-b72b-536a55eb34a9 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:17.137897 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af069dac-9071-444a-b72b-536a55eb34a9 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:17.138186 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af069dac-9071-444a-b72b-536a55eb34a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:17.183824 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-af069dac-9071-444a-b72b-536a55eb34a9 None None] MFA Rules not processed for user `fcba07a5d5a84bb4b1123d07192ad2e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:17.187626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-af069dac-9071-444a-b72b-536a55eb34a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:17.275924 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1134/2256] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:17.447525 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07fab98-8796-4f7d-94eb-508d83d42da6 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:17.448904 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a07fab98-8796-4f7d-94eb-508d83d42da6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:17.526183 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07fab98-8796-4f7d-94eb-508d83d42da6 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:17.530005 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1979474-f46d-43c0-8b36-fc326b7fb6ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:17.530735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1979474-f46d-43c0-8b36-fc326b7fb6ca service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:17.531142 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1979474-f46d-43c0-8b36-fc326b7fb6ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:17.531426 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1979474-f46d-43c0-8b36-fc326b7fb6ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:17.542002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1979474-f46d-43c0-8b36-fc326b7fb6ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:17.542811 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1979474-f46d-43c0-8b36-fc326b7fb6ca service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:17.555079 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1123/2257] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:18.487961 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:19.711187 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af069dac-9071-444a-b72b-536a55eb34a9 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:19.712633 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-af069dac-9071-444a-b72b-536a55eb34a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:19.788053 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af069dac-9071-444a-b72b-536a55eb34a9 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:19.792802 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b860b85-26d9-475c-9251-0247f96a38b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:19.797362 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b860b85-26d9-475c-9251-0247f96a38b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:19.797362 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b860b85-26d9-475c-9251-0247f96a38b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:19.797362 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b860b85-26d9-475c-9251-0247f96a38b1 admin admin] PATH_INFO: `/v3/users/70cf41938b14463096a18db37ec16400` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:19.797362 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b860b85-26d9-475c-9251-0247f96a38b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70cf41938b14463096a18db37ec16400)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:19.797724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b860b85-26d9-475c-9251-0247f96a38b1 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:19.816927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:19.983644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3b860b85-26d9-475c-9251-0247f96a38b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70cf41938b14463096a18db37ec16400'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:20.081349 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3b860b85-26d9-475c-9251-0247f96a38b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70cf41938b14463096a18db37ec16400'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:20.106442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3b860b85-26d9-475c-9251-0247f96a38b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70cf41938b14463096a18db37ec16400'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:20.107321 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3b860b85-26d9-475c-9251-0247f96a38b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70cf41938b14463096a18db37ec16400'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:20.115694 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3b860b85-26d9-475c-9251-0247f96a38b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70cf41938b14463096a18db37ec16400'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:20.122286 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1135/2258] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:37:19 2022] DELETE /identity/v3/users/70cf41938b14463096a18db37ec16400 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:20.127515 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68714b47-d511-4c48-bd3f-25cc242632fb None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:20.128192 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68714b47-d511-4c48-bd3f-25cc242632fb None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:20.128573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68714b47-d511-4c48-bd3f-25cc242632fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:20.146213 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-68714b47-d511-4c48-bd3f-25cc242632fb None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:20.147451 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-68714b47-d511-4c48-bd3f-25cc242632fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:20.209004 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1124/2259] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:37:20.224071 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b860b85-26d9-475c-9251-0247f96a38b1 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:20.226206 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3b860b85-26d9-475c-9251-0247f96a38b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:20.283513 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b860b85-26d9-475c-9251-0247f96a38b1 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:20.293765 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa06ac1-87db-4ff9-bcd6-6c55c5823cec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:20.294435 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa06ac1-87db-4ff9-bcd6-6c55c5823cec service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:20.294715 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa06ac1-87db-4ff9-bcd6-6c55c5823cec service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:20.295022 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa06ac1-87db-4ff9-bcd6-6c55c5823cec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:20.314008 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa06ac1-87db-4ff9-bcd6-6c55c5823cec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:20.314794 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa06ac1-87db-4ff9-bcd6-6c55c5823cec service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:20.327993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1136/2260] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:21.194231 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68714b47-d511-4c48-bd3f-25cc242632fb None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:21.196545 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-68714b47-d511-4c48-bd3f-25cc242632fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:21.219980 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68714b47-d511-4c48-bd3f-25cc242632fb None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:21.226542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bf6c5-b17e-4aef-8ffc-39b195be209b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:21.227376 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bf6c5-b17e-4aef-8ffc-39b195be209b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:21.227616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bf6c5-b17e-4aef-8ffc-39b195be209b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:21.227922 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bf6c5-b17e-4aef-8ffc-39b195be209b admin admin] PATH_INFO: `/v3/projects/1b85c126682b4cfbaf6963c88cae9dfd` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:21.229262 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856bf6c5-b17e-4aef-8ffc-39b195be209b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b85c126682b4cfbaf6963c88cae9dfd)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:21.231736 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856bf6c5-b17e-4aef-8ffc-39b195be209b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:21.381134 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-856bf6c5-b17e-4aef-8ffc-39b195be209b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b85c126682b4cfbaf6963c88cae9dfd', 'request_id': 'req-856bf6c5-b17e-4aef-8ffc-39b195be209b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:21.388352 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa06ac1-87db-4ff9-bcd6-6c55c5823cec service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:21.389983 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8fa06ac1-87db-4ff9-bcd6-6c55c5823cec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:21.404234 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa06ac1-87db-4ff9-bcd6-6c55c5823cec service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:21.422053 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:21.464419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5db9d8-6fb3-4f7e-991a-a446a236ba4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:21.465301 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5db9d8-6fb3-4f7e-991a-a446a236ba4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:21.465594 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5db9d8-6fb3-4f7e-991a-a446a236ba4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:21.465891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5db9d8-6fb3-4f7e-991a-a446a236ba4a admin admin] PATH_INFO: `/v3/users/7595af6e709e4538b3bb2d13a6461815` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:21.467121 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5db9d8-6fb3-4f7e-991a-a446a236ba4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7595af6e709e4538b3bb2d13a6461815)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:21.469567 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5db9d8-6fb3-4f7e-991a-a446a236ba4a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:21.470474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-856bf6c5-b17e-4aef-8ffc-39b195be209b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b85c126682b4cfbaf6963c88cae9dfd', 'request_id': 'req-856bf6c5-b17e-4aef-8ffc-39b195be209b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:21.479648 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-856bf6c5-b17e-4aef-8ffc-39b195be209b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b85c126682b4cfbaf6963c88cae9dfd', 'request_id': 'req-856bf6c5-b17e-4aef-8ffc-39b195be209b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:21.492136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-856bf6c5-b17e-4aef-8ffc-39b195be209b admin admin] The token cache is being invalidate because project 1b85c126682b4cfbaf6963c88cae9dfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:37:21.492788 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-856bf6c5-b17e-4aef-8ffc-39b195be209b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:21.493623 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-856bf6c5-b17e-4aef-8ffc-39b195be209b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:21.495780 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1125/2261] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:37:21 2022] DELETE /identity/v3/projects/1b85c126682b4cfbaf6963c88cae9dfd => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:21.499929 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bf6c5-b17e-4aef-8ffc-39b195be209b admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:21.501276 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-856bf6c5-b17e-4aef-8ffc-39b195be209b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:21.514460 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bf6c5-b17e-4aef-8ffc-39b195be209b admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:21.519426 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abf6137-193e-44a0-a5dc-05149e0d1bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:21.520459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf6137-193e-44a0-a5dc-05149e0d1bc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:21.521006 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf6137-193e-44a0-a5dc-05149e0d1bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:21.521845 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf6137-193e-44a0-a5dc-05149e0d1bc4 admin admin] PATH_INFO: `/v3/users/8ba205d768b94a90b0a1d26225501d97` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:21.523068 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abf6137-193e-44a0-a5dc-05149e0d1bc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ba205d768b94a90b0a1d26225501d97)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:21.524981 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abf6137-193e-44a0-a5dc-05149e0d1bc4 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:21.688286 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4d5db9d8-6fb3-4f7e-991a-a446a236ba4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7595af6e709e4538b3bb2d13a6461815'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:21.766111 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6abf6137-193e-44a0-a5dc-05149e0d1bc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ba205d768b94a90b0a1d26225501d97'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:21.778977 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4d5db9d8-6fb3-4f7e-991a-a446a236ba4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7595af6e709e4538b3bb2d13a6461815'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:21.792149 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4d5db9d8-6fb3-4f7e-991a-a446a236ba4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7595af6e709e4538b3bb2d13a6461815'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:21.792864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4d5db9d8-6fb3-4f7e-991a-a446a236ba4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7595af6e709e4538b3bb2d13a6461815'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:21.799642 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4d5db9d8-6fb3-4f7e-991a-a446a236ba4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7595af6e709e4538b3bb2d13a6461815'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:21.806056 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1137/2262] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:37:21 2022] DELETE /identity/v3/users/7595af6e709e4538b3bb2d13a6461815 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:21.810612 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b5c9a-8e62-4dc8-93f4-10d33dbc43f7 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:21.812157 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b5c9a-8e62-4dc8-93f4-10d33dbc43f7 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:21.812157 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b5c9a-8e62-4dc8-93f4-10d33dbc43f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:21.830320 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-ba6b5c9a-8e62-4dc8-93f4-10d33dbc43f7 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:21.831452 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ba6b5c9a-8e62-4dc8-93f4-10d33dbc43f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:21.906935 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1138/2263] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:37:21.917532 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6abf6137-193e-44a0-a5dc-05149e0d1bc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ba205d768b94a90b0a1d26225501d97'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:21.918022 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b5c9a-8e62-4dc8-93f4-10d33dbc43f7 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:21.918996 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6abf6137-193e-44a0-a5dc-05149e0d1bc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ba205d768b94a90b0a1d26225501d97'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:21.919775 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ba6b5c9a-8e62-4dc8-93f4-10d33dbc43f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:21.929651 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6abf6137-193e-44a0-a5dc-05149e0d1bc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ba205d768b94a90b0a1d26225501d97'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:21.933833 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6abf6137-193e-44a0-a5dc-05149e0d1bc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ba205d768b94a90b0a1d26225501d97'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:21.960349 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1126/2264] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:37:21 2022] DELETE /identity/v3/users/8ba205d768b94a90b0a1d26225501d97 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:37:22.007419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b5c9a-8e62-4dc8-93f4-10d33dbc43f7 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:22.101714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb2b47-b7e9-4db9-9d7d-6b367b865ccf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:22.102359 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb2b47-b7e9-4db9-9d7d-6b367b865ccf service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:22.102644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb2b47-b7e9-4db9-9d7d-6b367b865ccf service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:22.102935 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eb2b47-b7e9-4db9-9d7d-6b367b865ccf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:22.105411 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f9eb2b47-b7e9-4db9-9d7d-6b367b865ccf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:22.234737 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb2b47-b7e9-4db9-9d7d-6b367b865ccf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:22.236334 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eb2b47-b7e9-4db9-9d7d-6b367b865ccf service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:22.251129 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1139/2265] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:22.453427 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:23.130911 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abf6137-193e-44a0-a5dc-05149e0d1bc4 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:23.132664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6abf6137-193e-44a0-a5dc-05149e0d1bc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:23.151404 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abf6137-193e-44a0-a5dc-05149e0d1bc4 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:23.160229 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d8088-3221-4634-b682-6e174c87ce50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:23.163580 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d8088-3221-4634-b682-6e174c87ce50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:23.164181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d8088-3221-4634-b682-6e174c87ce50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:23.164628 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d8088-3221-4634-b682-6e174c87ce50 admin admin] PATH_INFO: `/v3/projects/3f80d9e7b9964069a5b143e40135a665` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:23.168931 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d8088-3221-4634-b682-6e174c87ce50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f80d9e7b9964069a5b143e40135a665)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:23.173954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d8088-3221-4634-b682-6e174c87ce50 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:23.316945 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bf2d8088-3221-4634-b682-6e174c87ce50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f80d9e7b9964069a5b143e40135a665', 'request_id': 'req-bf2d8088-3221-4634-b682-6e174c87ce50'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:23.476185 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bf2d8088-3221-4634-b682-6e174c87ce50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f80d9e7b9964069a5b143e40135a665', 'request_id': 'req-bf2d8088-3221-4634-b682-6e174c87ce50'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:23.486261 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bf2d8088-3221-4634-b682-6e174c87ce50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f80d9e7b9964069a5b143e40135a665', 'request_id': 'req-bf2d8088-3221-4634-b682-6e174c87ce50'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:23.499594 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb2b47-b7e9-4db9-9d7d-6b367b865ccf service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:23.501642 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f9eb2b47-b7e9-4db9-9d7d-6b367b865ccf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:23.507286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bf2d8088-3221-4634-b682-6e174c87ce50 admin admin] The token cache is being invalidate because project 3f80d9e7b9964069a5b143e40135a665 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:37:23.507458 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bf2d8088-3221-4634-b682-6e174c87ce50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:23.508267 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bf2d8088-3221-4634-b682-6e174c87ce50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:23.513525 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1127/2266] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:37:23 2022] DELETE /identity/v3/projects/3f80d9e7b9964069a5b143e40135a665 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:23.515262 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb79eb-3763-473f-a751-f56e76bfb451 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:23.522480 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eb2b47-b7e9-4db9-9d7d-6b367b865ccf service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:23.526358 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb79eb-3763-473f-a751-f56e76bfb451 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:23.526566 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb79eb-3763-473f-a751-f56e76bfb451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:23.547613 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-b1fb79eb-3763-473f-a751-f56e76bfb451 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:23.549299 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b1fb79eb-3763-473f-a751-f56e76bfb451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:23.650902 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf31db-76ce-4e63-8ced-1657ec736763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:23.654724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf31db-76ce-4e63-8ced-1657ec736763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:23.655287 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf31db-76ce-4e63-8ced-1657ec736763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:23.655797 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf31db-76ce-4e63-8ced-1657ec736763 admin admin] PATH_INFO: `/v3/projects/41295e76fc1c400db439f17d1cf64d95` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:23.660041 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf31db-76ce-4e63-8ced-1657ec736763 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41295e76fc1c400db439f17d1cf64d95)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:23.665872 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf31db-76ce-4e63-8ced-1657ec736763 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:23.669527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:23.701347 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1128/2267] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:23.709651 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880287c-d1cb-47b2-a6a0-ee9ffc1a18dc None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:23.710016 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880287c-d1cb-47b2-a6a0-ee9ffc1a18dc None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:23.710366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880287c-d1cb-47b2-a6a0-ee9ffc1a18dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:23.735419 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-3880287c-d1cb-47b2-a6a0-ee9ffc1a18dc None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:23.737070 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3880287c-d1cb-47b2-a6a0-ee9ffc1a18dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:23.753050 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1129/2268] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:23.765569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880287c-d1cb-47b2-a6a0-ee9ffc1a18dc None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:23.790290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880287c-d1cb-47b2-a6a0-ee9ffc1a18dc None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:23.807015 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fec446-3598-4ec6-aa05-734e02a6c7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:23.809945 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fec446-3598-4ec6-aa05-734e02a6c7e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:23.809945 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fec446-3598-4ec6-aa05-734e02a6c7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:23.812364 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fec446-3598-4ec6-aa05-734e02a6c7e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:23.813303 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fec446-3598-4ec6-aa05-734e02a6c7e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:37:23.813524 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fec446-3598-4ec6-aa05-734e02a6c7e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:23.817374 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fec446-3598-4ec6-aa05-734e02a6c7e1 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:23.839724 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1130/2269] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:37:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:37:23.844864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fec446-3598-4ec6-aa05-734e02a6c7e1 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:23.860950 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fec446-3598-4ec6-aa05-734e02a6c7e1 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:23.869170 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-41cf31db-76ce-4e63-8ced-1657ec736763 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41295e76fc1c400db439f17d1cf64d95', 'request_id': 'req-41cf31db-76ce-4e63-8ced-1657ec736763'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:23.968793 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-41cf31db-76ce-4e63-8ced-1657ec736763 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41295e76fc1c400db439f17d1cf64d95', 'request_id': 'req-41cf31db-76ce-4e63-8ced-1657ec736763'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:23.985598 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-41cf31db-76ce-4e63-8ced-1657ec736763 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41295e76fc1c400db439f17d1cf64d95', 'request_id': 'req-41cf31db-76ce-4e63-8ced-1657ec736763'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:24.021929 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deae875-daba-4164-8d0f-323dcab453de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:24.022215 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deae875-daba-4164-8d0f-323dcab453de admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:24.022677 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deae875-daba-4164-8d0f-323dcab453de admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:24.022677 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deae875-daba-4164-8d0f-323dcab453de admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:24.023426 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deae875-daba-4164-8d0f-323dcab453de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:24.025381 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deae875-daba-4164-8d0f-323dcab453de admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:24.030678 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-41cf31db-76ce-4e63-8ced-1657ec736763 admin admin] The token cache is being invalidate because project 41295e76fc1c400db439f17d1cf64d95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:37:24.032404 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-41cf31db-76ce-4e63-8ced-1657ec736763 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:24.033204 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-41cf31db-76ce-4e63-8ced-1657ec736763 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:24.037662 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1140/2270] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:37:23 2022] DELETE /identity/v3/projects/41295e76fc1c400db439f17d1cf64d95 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:24.233441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf31db-76ce-4e63-8ced-1657ec736763 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:24.233441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-41cf31db-76ce-4e63-8ced-1657ec736763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:24.242200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1131/2271] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:37:23 2022] POST /identity/v3/projects => generated 383 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:24.260810 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf31db-76ce-4e63-8ced-1657ec736763 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:24.410743 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:24.418960 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f927d01-cf49-49b8-920e-830b5451a9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:24.419622 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f927d01-cf49-49b8-920e-830b5451a9bd admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:24.419872 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f927d01-cf49-49b8-920e-830b5451a9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:24.420144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f927d01-cf49-49b8-920e-830b5451a9bd admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:24.420849 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f927d01-cf49-49b8-920e-830b5451a9bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:24.422783 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f927d01-cf49-49b8-920e-830b5451a9bd admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:24.549221 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1141/2272] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:37:24 2022] POST /identity/v3/users => generated 292 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:37:24.556337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deae875-daba-4164-8d0f-323dcab453de admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:24.570222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deae875-daba-4164-8d0f-323dcab453de admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:24.584168 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2ad27-3ec4-445b-908e-f748d74de337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:24.585166 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2ad27-3ec4-445b-908e-f748d74de337 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:24.585684 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2ad27-3ec4-445b-908e-f748d74de337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:24.587276 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad2ad27-3ec4-445b-908e-f748d74de337 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:24.589521 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2ad27-3ec4-445b-908e-f748d74de337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:24.591431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad2ad27-3ec4-445b-908e-f748d74de337 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:24.620191 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f927d01-cf49-49b8-920e-830b5451a9bd admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:24.621048 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1132/2273] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:24.637287 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f927d01-cf49-49b8-920e-830b5451a9bd admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:24.654222 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50615300-2fce-4c07-84d4-0a0aa815d146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:24.654959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50615300-2fce-4c07-84d4-0a0aa815d146 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:24.655243 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50615300-2fce-4c07-84d4-0a0aa815d146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:24.655525 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50615300-2fce-4c07-84d4-0a0aa815d146 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:24.656263 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50615300-2fce-4c07-84d4-0a0aa815d146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:24.657888 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50615300-2fce-4c07-84d4-0a0aa815d146 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:24.694075 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1142/2274] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:24 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:24.698582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2ad27-3ec4-445b-908e-f748d74de337 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:24.726521 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad2ad27-3ec4-445b-908e-f748d74de337 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:24.742068 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a4f8f-734f-48ef-b172-e4eb6ab3dcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:24.743955 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a4f8f-734f-48ef-b172-e4eb6ab3dcbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:24.745019 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a4f8f-734f-48ef-b172-e4eb6ab3dcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:24.745524 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4a4f8f-734f-48ef-b172-e4eb6ab3dcbb admin admin] PATH_INFO: `/v3/projects/795234d85b6c4fd5b464998bedc59d82/users/e63f1516bc2948308ac2c4a15897d114/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:24.784472 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a4f8f-734f-48ef-b172-e4eb6ab3dcbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=795234d85b6c4fd5b464998bedc59d82, user_id=e63f1516bc2948308ac2c4a15897d114, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:24.790907 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4a4f8f-734f-48ef-b172-e4eb6ab3dcbb admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:24.879716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:24.922263 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1133/2275] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:37:24 2022] PUT /identity/v3/projects/795234d85b6c4fd5b464998bedc59d82/users/e63f1516bc2948308ac2c4a15897d114/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:24.936787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b255efb-aba2-4f42-a9d8-0df4a28da7d3 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:24.937072 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b255efb-aba2-4f42-a9d8-0df4a28da7d3 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:24.937598 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b255efb-aba2-4f42-a9d8-0df4a28da7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:24.990410 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-1b255efb-aba2-4f42-a9d8-0df4a28da7d3 None None] MFA Rules not processed for user `e63f1516bc2948308ac2c4a15897d114`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:24.991968 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1b255efb-aba2-4f42-a9d8-0df4a28da7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:25.122168 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1143/2276] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:25.128682 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a4f8f-734f-48ef-b172-e4eb6ab3dcbb admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:25.276832 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4a4f8f-734f-48ef-b172-e4eb6ab3dcbb admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:25.282991 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c74c2-d611-49e5-9b07-07a3b02c7704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:25.284001 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c74c2-d611-49e5-9b07-07a3b02c7704 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:25.285290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c74c2-d611-49e5-9b07-07a3b02c7704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:25.285813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c74c2-d611-49e5-9b07-07a3b02c7704 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:25.287683 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c74c2-d611-49e5-9b07-07a3b02c7704 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:25.290698 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c74c2-d611-49e5-9b07-07a3b02c7704 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:25.362868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a83956-f3a0-48e0-b3e3-da06f5c27417 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:25.363224 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a83956-f3a0-48e0-b3e3-da06f5c27417 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:25.363419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a83956-f3a0-48e0-b3e3-da06f5c27417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:25.395409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-c5a83956-f3a0-48e0-b3e3-da06f5c27417 None None] MFA Rules not processed for user `07c9688ca035484ba5341859574c2799`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:25.396941 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c5a83956-f3a0-48e0-b3e3-da06f5c27417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:25.418188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1134/2277] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:37:25 2022] POST /identity/v3/projects => generated 381 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:37:25.422570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c74c2-d611-49e5-9b07-07a3b02c7704 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:25.524179 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c74c2-d611-49e5-9b07-07a3b02c7704 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:25.531336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6993c-1d5c-47c5-8ff9-4eab0b6ef9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:25.532720 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6993c-1d5c-47c5-8ff9-4eab0b6ef9f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:25.533105 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6993c-1d5c-47c5-8ff9-4eab0b6ef9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:25.533481 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6993c-1d5c-47c5-8ff9-4eab0b6ef9f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:25.534390 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6993c-1d5c-47c5-8ff9-4eab0b6ef9f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:25.536635 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d6993c-1d5c-47c5-8ff9-4eab0b6ef9f6 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:25.550048 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1144/2278] 10.222.0.24 () {58 vars in 1009 bytes} [Fri May 27 03:37:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:37:25.610230 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a83956-f3a0-48e0-b3e3-da06f5c27417 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:25.611858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c5a83956-f3a0-48e0-b3e3-da06f5c27417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:25.644507 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1135/2279] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:37:25 2022] POST /identity/v3/users => generated 297 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:37:25.648913 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6993c-1d5c-47c5-8ff9-4eab0b6ef9f6 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:25.665740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6993c-1d5c-47c5-8ff9-4eab0b6ef9f6 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:25.673187 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f41c2-192d-4d87-8d42-81c0fe01e408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:25.674152 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f41c2-192d-4d87-8d42-81c0fe01e408 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:25.675244 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f41c2-192d-4d87-8d42-81c0fe01e408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:25.675893 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f41c2-192d-4d87-8d42-81c0fe01e408 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:25.676932 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f41c2-192d-4d87-8d42-81c0fe01e408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:25.689931 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f41c2-192d-4d87-8d42-81c0fe01e408 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:25.698864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1136/2280] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:37:25.702910 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f41c2-192d-4d87-8d42-81c0fe01e408 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:25.725738 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f41c2-192d-4d87-8d42-81c0fe01e408 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:25.736718 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a83956-f3a0-48e0-b3e3-da06f5c27417 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:25.737195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06c8c1-83a5-4a48-86dd-2f590fa6e6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:25.738643 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06c8c1-83a5-4a48-86dd-2f590fa6e6e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:25.739161 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06c8c1-83a5-4a48-86dd-2f590fa6e6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:25.739639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06c8c1-83a5-4a48-86dd-2f590fa6e6e3 admin admin] PATH_INFO: `/v3/projects/436e3eb9752a472dbc349d07a1b388d3/users/a7971cc200ae4f0fb2c4a88de4567e00/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:25.757839 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06c8c1-83a5-4a48-86dd-2f590fa6e6e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=436e3eb9752a472dbc349d07a1b388d3, user_id=a7971cc200ae4f0fb2c4a88de4567e00, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:25.761017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06c8c1-83a5-4a48-86dd-2f590fa6e6e3 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:25.819026 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4561bc-c809-4721-a1c6-0f7eada345b4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d559cd7127490da84185f20684929d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:25.819756 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4561bc-c809-4721-a1c6-0f7eada345b4 service swift] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:25.820121 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4561bc-c809-4721-a1c6-0f7eada345b4 service swift] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:25.820462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4561bc-c809-4721-a1c6-0f7eada345b4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:25.849340 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1137/2281] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:37:25 2022] PUT /identity/v3/projects/436e3eb9752a472dbc349d07a1b388d3/users/a7971cc200ae4f0fb2c4a88de4567e00/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:37:25.850197 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06c8c1-83a5-4a48-86dd-2f590fa6e6e3 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:25.968310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4561bc-c809-4721-a1c6-0f7eada345b4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:25.972853 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4561bc-c809-4721-a1c6-0f7eada345b4 service swift] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:25.984443 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06c8c1-83a5-4a48-86dd-2f590fa6e6e3 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:25.993771 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1145/2282] 10.222.0.24 () {60 vars in 1384 bytes} [Fri May 27 03:37:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 03:37:25.997974 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87baf3d-0c6f-4107-9f06-36747249a0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:25.998591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87baf3d-0c6f-4107-9f06-36747249a0a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:25.998821 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87baf3d-0c6f-4107-9f06-36747249a0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:25.999207 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87baf3d-0c6f-4107-9f06-36747249a0a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:25.999856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87baf3d-0c6f-4107-9f06-36747249a0a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:26.001493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87baf3d-0c6f-4107-9f06-36747249a0a9 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:26.017953 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:26.040736 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1138/2283] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:25 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:26.045461 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4561bc-c809-4721-a1c6-0f7eada345b4 service swift] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:26.065489 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4561bc-c809-4721-a1c6-0f7eada345b4 service swift] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:26.078073 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05133c61-7a32-4118-ad10-d1bde5a9c9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:26.082710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05133c61-7a32-4118-ad10-d1bde5a9c9c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:26.083237 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05133c61-7a32-4118-ad10-d1bde5a9c9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:26.083686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05133c61-7a32-4118-ad10-d1bde5a9c9c4 admin admin] PATH_INFO: `/v3/projects/436e3eb9752a472dbc349d07a1b388d3/users/a7971cc200ae4f0fb2c4a88de4567e00/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:26.086349 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05133c61-7a32-4118-ad10-d1bde5a9c9c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=436e3eb9752a472dbc349d07a1b388d3, user_id=a7971cc200ae4f0fb2c4a88de4567e00, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:26.109180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05133c61-7a32-4118-ad10-d1bde5a9c9c4 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:26.139015 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1146/2284] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:37:26 2022] PUT /identity/v3/projects/436e3eb9752a472dbc349d07a1b388d3/users/a7971cc200ae4f0fb2c4a88de4567e00/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:37:26.150783 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b900a-6b38-4b86-828a-06f30c896a2f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:26.151231 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b900a-6b38-4b86-828a-06f30c896a2f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:26.151402 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b900a-6b38-4b86-828a-06f30c896a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:26.204043 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-122b900a-6b38-4b86-828a-06f30c896a2f None None] MFA Rules not processed for user `a7971cc200ae4f0fb2c4a88de4567e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:26.205289 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-122b900a-6b38-4b86-828a-06f30c896a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:26.313818 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1139/2285] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:26 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:26.324428 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05133c61-7a32-4118-ad10-d1bde5a9c9c4 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:26.326687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-05133c61-7a32-4118-ad10-d1bde5a9c9c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:26.428710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05133c61-7a32-4118-ad10-d1bde5a9c9c4 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:26.435293 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29436e7-96d4-4e0d-8f27-8c57c5aed0ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:26.436559 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29436e7-96d4-4e0d-8f27-8c57c5aed0ed service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:26.436978 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29436e7-96d4-4e0d-8f27-8c57c5aed0ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:26.437262 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29436e7-96d4-4e0d-8f27-8c57c5aed0ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:26.447764 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29436e7-96d4-4e0d-8f27-8c57c5aed0ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:26.449930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29436e7-96d4-4e0d-8f27-8c57c5aed0ed service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:26.464651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1147/2286] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:26 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:26.466211 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:26.745465 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:26.779567 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b900a-6b38-4b86-828a-06f30c896a2f None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:26.790612 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b900a-6b38-4b86-828a-06f30c896a2f None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:26.794984 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290b09d-9295-43d8-90ad-74cb81019b9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:26.795887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290b09d-9295-43d8-90ad-74cb81019b9f service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:26.796325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290b09d-9295-43d8-90ad-74cb81019b9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:26.796773 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290b09d-9295-43d8-90ad-74cb81019b9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:26.895392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290b09d-9295-43d8-90ad-74cb81019b9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:26.897394 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290b09d-9295-43d8-90ad-74cb81019b9f service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:26.907162 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:26.936612 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1140/2287] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:26.967808 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:28.295921 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:28.524254 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:29.922360 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:30.852235 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29436e7-96d4-4e0d-8f27-8c57c5aed0ed service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:30.915757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29436e7-96d4-4e0d-8f27-8c57c5aed0ed service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:30.921004 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4afbb-828c-48d2-9365-a45db58d5aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:30.921883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4afbb-828c-48d2-9365-a45db58d5aa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:30.922305 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4afbb-828c-48d2-9365-a45db58d5aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:30.922581 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a4afbb-828c-48d2-9365-a45db58d5aa6 admin admin] PATH_INFO: `/v3/users/e63f1516bc2948308ac2c4a15897d114` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:30.923915 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a4afbb-828c-48d2-9365-a45db58d5aa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e63f1516bc2948308ac2c4a15897d114)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:30.925926 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a4afbb-828c-48d2-9365-a45db58d5aa6 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:31.160640 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-58a4afbb-828c-48d2-9365-a45db58d5aa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e63f1516bc2948308ac2c4a15897d114'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:31.232807 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-58a4afbb-828c-48d2-9365-a45db58d5aa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e63f1516bc2948308ac2c4a15897d114'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:31.248977 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-58a4afbb-828c-48d2-9365-a45db58d5aa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e63f1516bc2948308ac2c4a15897d114'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:31.249721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-58a4afbb-828c-48d2-9365-a45db58d5aa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e63f1516bc2948308ac2c4a15897d114'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:31.259542 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-58a4afbb-828c-48d2-9365-a45db58d5aa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e63f1516bc2948308ac2c4a15897d114'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:31.267492 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1148/2288] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:37:30 2022] DELETE /identity/v3/users/e63f1516bc2948308ac2c4a15897d114 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:31.272407 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ac7c2-dd87-4ee2-ad06-bc2330884d29 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:31.272891 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ac7c2-dd87-4ee2-ad06-bc2330884d29 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:31.272979 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ac7c2-dd87-4ee2-ad06-bc2330884d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:31.303163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-b49ac7c2-dd87-4ee2-ad06-bc2330884d29 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:31.304458 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b49ac7c2-dd87-4ee2-ad06-bc2330884d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:31.348584 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:31.413591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1141/2289] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:31.423443 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4afbb-828c-48d2-9365-a45db58d5aa6 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:31.428013 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-58a4afbb-828c-48d2-9365-a45db58d5aa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:31.509503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a4afbb-828c-48d2-9365-a45db58d5aa6 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:31.520946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9bb42f-6475-433f-a8b1-c852db36199d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:31.524920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bb42f-6475-433f-a8b1-c852db36199d service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:31.524920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bb42f-6475-433f-a8b1-c852db36199d service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:31.524920 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bb42f-6475-433f-a8b1-c852db36199d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:31.548126 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9bb42f-6475-433f-a8b1-c852db36199d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:31.550870 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9bb42f-6475-433f-a8b1-c852db36199d service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:31.561531 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1149/2290] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:32.174238 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ac7c2-dd87-4ee2-ad06-bc2330884d29 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:32.175802 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b49ac7c2-dd87-4ee2-ad06-bc2330884d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:32.188117 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9bb42f-6475-433f-a8b1-c852db36199d service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:32.189675 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ff9bb42f-6475-433f-a8b1-c852db36199d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:32.212080 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9bb42f-6475-433f-a8b1-c852db36199d service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:32.213686 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ac7c2-dd87-4ee2-ad06-bc2330884d29 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:32.218502 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3736769-b6dd-4849-9f81-bec4d1eec3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:32.219402 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b902f25-6e45-4086-a699-891cfb22590a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:32.219794 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3736769-b6dd-4849-9f81-bec4d1eec3bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:32.220290 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3736769-b6dd-4849-9f81-bec4d1eec3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:32.220404 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b902f25-6e45-4086-a699-891cfb22590a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:32.220586 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b902f25-6e45-4086-a699-891cfb22590a admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:32.220961 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b902f25-6e45-4086-a699-891cfb22590a admin admin] PATH_INFO: `/v3/projects/795234d85b6c4fd5b464998bedc59d82` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:32.221100 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3736769-b6dd-4849-9f81-bec4d1eec3bb admin admin] PATH_INFO: `/v3/users/fcba07a5d5a84bb4b1123d07192ad2e1` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:32.222245 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b902f25-6e45-4086-a699-891cfb22590a admin admin] RBAC: Authorizing `identity:delete_project(project_id=795234d85b6c4fd5b464998bedc59d82)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:32.222594 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3736769-b6dd-4849-9f81-bec4d1eec3bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcba07a5d5a84bb4b1123d07192ad2e1)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:32.224513 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b902f25-6e45-4086-a699-891cfb22590a admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:32.224715 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3736769-b6dd-4849-9f81-bec4d1eec3bb admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:32.470043 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b3736769-b6dd-4849-9f81-bec4d1eec3bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcba07a5d5a84bb4b1123d07192ad2e1'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:32.477545 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0b902f25-6e45-4086-a699-891cfb22590a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '795234d85b6c4fd5b464998bedc59d82', 'request_id': 'req-0b902f25-6e45-4086-a699-891cfb22590a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:32.541291 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b3736769-b6dd-4849-9f81-bec4d1eec3bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcba07a5d5a84bb4b1123d07192ad2e1'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:32.559902 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b3736769-b6dd-4849-9f81-bec4d1eec3bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcba07a5d5a84bb4b1123d07192ad2e1'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:32.560895 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b3736769-b6dd-4849-9f81-bec4d1eec3bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcba07a5d5a84bb4b1123d07192ad2e1'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:32.570558 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-b3736769-b6dd-4849-9f81-bec4d1eec3bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcba07a5d5a84bb4b1123d07192ad2e1'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:32.584175 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1150/2291] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:37:32 2022] DELETE /identity/v3/users/fcba07a5d5a84bb4b1123d07192ad2e1 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:37:32.588451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d140cbb-a998-4dcb-9821-912c86eb581f None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:32.589034 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d140cbb-a998-4dcb-9821-912c86eb581f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:32.589522 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d140cbb-a998-4dcb-9821-912c86eb581f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:32.612984 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-6d140cbb-a998-4dcb-9821-912c86eb581f None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:32.614317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6d140cbb-a998-4dcb-9821-912c86eb581f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:32.649721 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0b902f25-6e45-4086-a699-891cfb22590a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '795234d85b6c4fd5b464998bedc59d82', 'request_id': 'req-0b902f25-6e45-4086-a699-891cfb22590a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:32.656680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0b902f25-6e45-4086-a699-891cfb22590a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '795234d85b6c4fd5b464998bedc59d82', 'request_id': 'req-0b902f25-6e45-4086-a699-891cfb22590a'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:32.687278 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1151/2292] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:37:32.696181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0b902f25-6e45-4086-a699-891cfb22590a admin admin] The token cache is being invalidate because project 795234d85b6c4fd5b464998bedc59d82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:37:32.701243 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d140cbb-a998-4dcb-9821-912c86eb581f None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:32.701493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0b902f25-6e45-4086-a699-891cfb22590a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:32.701493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0b902f25-6e45-4086-a699-891cfb22590a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:32.702083 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6d140cbb-a998-4dcb-9821-912c86eb581f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:32.703736 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1142/2293] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:37:32 2022] DELETE /identity/v3/projects/795234d85b6c4fd5b464998bedc59d82 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:32.707613 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b902f25-6e45-4086-a699-891cfb22590a admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:32.709401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-0b902f25-6e45-4086-a699-891cfb22590a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:32.810564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d140cbb-a998-4dcb-9821-912c86eb581f None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:32.815033 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a36c87-1241-44e4-a2c7-b148d595cb0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:32.815207 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b902f25-6e45-4086-a699-891cfb22590a admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:32.815890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a36c87-1241-44e4-a2c7-b148d595cb0d service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:32.816357 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a36c87-1241-44e4-a2c7-b148d595cb0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:32.816731 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a36c87-1241-44e4-a2c7-b148d595cb0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:32.819392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f7a36c87-1241-44e4-a2c7-b148d595cb0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:32.821001 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635a17d-493c-4bfd-9cd3-dc62da1278d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:32.821979 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635a17d-493c-4bfd-9cd3-dc62da1278d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:32.822253 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635a17d-493c-4bfd-9cd3-dc62da1278d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:32.822694 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635a17d-493c-4bfd-9cd3-dc62da1278d6 admin admin] PATH_INFO: `/v3/users/a7971cc200ae4f0fb2c4a88de4567e00` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:32.835665 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635a17d-493c-4bfd-9cd3-dc62da1278d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7971cc200ae4f0fb2c4a88de4567e00)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:32.840949 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a36c87-1241-44e4-a2c7-b148d595cb0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:32.841108 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635a17d-493c-4bfd-9cd3-dc62da1278d6 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:32.841202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a36c87-1241-44e4-a2c7-b148d595cb0d service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:32.855192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1152/2294] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:32.861448 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:33.093708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-8635a17d-493c-4bfd-9cd3-dc62da1278d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7971cc200ae4f0fb2c4a88de4567e00'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:33.159203 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-8635a17d-493c-4bfd-9cd3-dc62da1278d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7971cc200ae4f0fb2c4a88de4567e00'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:33.160117 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-8635a17d-493c-4bfd-9cd3-dc62da1278d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7971cc200ae4f0fb2c4a88de4567e00'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:33.173722 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-8635a17d-493c-4bfd-9cd3-dc62da1278d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7971cc200ae4f0fb2c4a88de4567e00'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:33.174597 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-8635a17d-493c-4bfd-9cd3-dc62da1278d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7971cc200ae4f0fb2c4a88de4567e00'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:33.216182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1143/2295] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:37:32 2022] DELETE /identity/v3/users/a7971cc200ae4f0fb2c4a88de4567e00 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:33.468159 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a36c87-1241-44e4-a2c7-b148d595cb0d service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:33.470416 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f7a36c87-1241-44e4-a2c7-b148d595cb0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:33.503377 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a36c87-1241-44e4-a2c7-b148d595cb0d service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:33.573332 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8abe6f-eb22-40d8-81aa-aab61caae3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:33.574270 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8abe6f-eb22-40d8-81aa-aab61caae3f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:33.574557 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8abe6f-eb22-40d8-81aa-aab61caae3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:33.574831 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8abe6f-eb22-40d8-81aa-aab61caae3f0 admin admin] PATH_INFO: `/v3/projects/7a486708a40b42ceb41394312e4da42e` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:33.576145 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8abe6f-eb22-40d8-81aa-aab61caae3f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a486708a40b42ceb41394312e4da42e)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:33.578566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8abe6f-eb22-40d8-81aa-aab61caae3f0 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:33.742281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0f8abe6f-eb22-40d8-81aa-aab61caae3f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a486708a40b42ceb41394312e4da42e', 'request_id': 'req-0f8abe6f-eb22-40d8-81aa-aab61caae3f0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:33.839259 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0f8abe6f-eb22-40d8-81aa-aab61caae3f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a486708a40b42ceb41394312e4da42e', 'request_id': 'req-0f8abe6f-eb22-40d8-81aa-aab61caae3f0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:33.840413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:33.849664 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0f8abe6f-eb22-40d8-81aa-aab61caae3f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a486708a40b42ceb41394312e4da42e', 'request_id': 'req-0f8abe6f-eb22-40d8-81aa-aab61caae3f0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:33.875514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0f8abe6f-eb22-40d8-81aa-aab61caae3f0 admin admin] The token cache is being invalidate because project 7a486708a40b42ceb41394312e4da42e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:37:33.875872 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0f8abe6f-eb22-40d8-81aa-aab61caae3f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:33.877637 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-0f8abe6f-eb22-40d8-81aa-aab61caae3f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:33.883061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1153/2296] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:37:33 2022] DELETE /identity/v3/projects/7a486708a40b42ceb41394312e4da42e => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:33.887978 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c851bd-d87b-4fd6-afd1-827acc13e475 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:33.904753 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c851bd-d87b-4fd6-afd1-827acc13e475 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:33.904753 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c851bd-d87b-4fd6-afd1-827acc13e475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:33.914898 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-21c851bd-d87b-4fd6-afd1-827acc13e475 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:33.916164 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-21c851bd-d87b-4fd6-afd1-827acc13e475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:33.986983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1144/2297] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:33.996274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f5688-cb67-4259-85ef-07aa78d04ed9 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:33.996956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f5688-cb67-4259-85ef-07aa78d04ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:33.997423 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f5688-cb67-4259-85ef-07aa78d04ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:34.017756 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-8d7f5688-cb67-4259-85ef-07aa78d04ed9 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:34.019127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8d7f5688-cb67-4259-85ef-07aa78d04ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:34.031847 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1154/2298] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:37:34.032844 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:34.033381 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c851bd-d87b-4fd6-afd1-827acc13e475 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:34.049026 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c851bd-d87b-4fd6-afd1-827acc13e475 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:34.052867 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f5688-cb67-4259-85ef-07aa78d04ed9 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:34.054181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8d7f5688-cb67-4259-85ef-07aa78d04ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:34.059438 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f3381-779e-4079-9311-de9f143c81ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:34.060434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f3381-779e-4079-9311-de9f143c81ce admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:34.060904 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f3381-779e-4079-9311-de9f143c81ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:34.061806 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f3381-779e-4079-9311-de9f143c81ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:34.063880 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f3381-779e-4079-9311-de9f143c81ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:37:34.064612 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f3381-779e-4079-9311-de9f143c81ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:34.067804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f3381-779e-4079-9311-de9f143c81ce admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:34.078664 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f5688-cb67-4259-85ef-07aa78d04ed9 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:34.088902 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e7bb9-6822-43cd-8a3e-d71c8df04d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:34.089927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e7bb9-6822-43cd-8a3e-d71c8df04d6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:34.090215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e7bb9-6822-43cd-8a3e-d71c8df04d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:34.090496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e7bb9-6822-43cd-8a3e-d71c8df04d6f admin admin] PATH_INFO: `/v3/projects/436e3eb9752a472dbc349d07a1b388d3` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:34.093442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e7bb9-6822-43cd-8a3e-d71c8df04d6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=436e3eb9752a472dbc349d07a1b388d3)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:34.094904 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1145/2299] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:37:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:37:34.099525 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e7bb9-6822-43cd-8a3e-d71c8df04d6f admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:34.127330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f3381-779e-4079-9311-de9f143c81ce admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:34.169005 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f3381-779e-4079-9311-de9f143c81ce admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:34.181059 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551a8d5-9b91-47da-974c-25bca9d741cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:34.181737 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551a8d5-9b91-47da-974c-25bca9d741cd admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:34.182018 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551a8d5-9b91-47da-974c-25bca9d741cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:34.182290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551a8d5-9b91-47da-974c-25bca9d741cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:34.183101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1551a8d5-9b91-47da-974c-25bca9d741cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:34.185570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1551a8d5-9b91-47da-974c-25bca9d741cd admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:34.313662 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-c06e7bb9-6822-43cd-8a3e-d71c8df04d6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '436e3eb9752a472dbc349d07a1b388d3', 'request_id': 'req-c06e7bb9-6822-43cd-8a3e-d71c8df04d6f'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:34.367570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1146/2300] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:37:34 2022] POST /identity/v3/projects => generated 385 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:34.368664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551a8d5-9b91-47da-974c-25bca9d741cd admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:34.387241 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-c06e7bb9-6822-43cd-8a3e-d71c8df04d6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '436e3eb9752a472dbc349d07a1b388d3', 'request_id': 'req-c06e7bb9-6822-43cd-8a3e-d71c8df04d6f'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:34.395460 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-c06e7bb9-6822-43cd-8a3e-d71c8df04d6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '436e3eb9752a472dbc349d07a1b388d3', 'request_id': 'req-c06e7bb9-6822-43cd-8a3e-d71c8df04d6f'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:34.413991 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-c06e7bb9-6822-43cd-8a3e-d71c8df04d6f admin admin] The token cache is being invalidate because project 436e3eb9752a472dbc349d07a1b388d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:37:34.414230 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-c06e7bb9-6822-43cd-8a3e-d71c8df04d6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:34.416146 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-c06e7bb9-6822-43cd-8a3e-d71c8df04d6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:34.418852 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1155/2301] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:37:34 2022] DELETE /identity/v3/projects/436e3eb9752a472dbc349d07a1b388d3 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:34.425772 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518cae8-09e2-4fac-bbb8-804b0ebba654 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:34.426237 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518cae8-09e2-4fac-bbb8-804b0ebba654 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:34.426676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518cae8-09e2-4fac-bbb8-804b0ebba654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:34.453631 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-e518cae8-09e2-4fac-bbb8-804b0ebba654 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:34.455357 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e518cae8-09e2-4fac-bbb8-804b0ebba654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:34.494946 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1551a8d5-9b91-47da-974c-25bca9d741cd admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:34.502227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd0cc05-9dcb-4af4-9f10-989aaac87faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:34.502987 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0cc05-9dcb-4af4-9f10-989aaac87faa admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:34.503343 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0cc05-9dcb-4af4-9f10-989aaac87faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:34.504252 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0cc05-9dcb-4af4-9f10-989aaac87faa admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:34.504995 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd0cc05-9dcb-4af4-9f10-989aaac87faa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:34.507364 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd0cc05-9dcb-4af4-9f10-989aaac87faa admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:34.540791 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1156/2302] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:34.548868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ae03d-d989-4cc6-9bd4-596f80a7cbc0 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:34.549154 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ae03d-d989-4cc6-9bd4-596f80a7cbc0 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:34.549438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ae03d-d989-4cc6-9bd4-596f80a7cbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:34.568485 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-fc7ae03d-d989-4cc6-9bd4-596f80a7cbc0 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:34.569725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-fc7ae03d-d989-4cc6-9bd4-596f80a7cbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:34.578305 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1157/2303] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:37:34.582603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7ae03d-d989-4cc6-9bd4-596f80a7cbc0 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:34.592938 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7ae03d-d989-4cc6-9bd4-596f80a7cbc0 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:34.598738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869d5af-c295-48f8-a30f-0697b72a6b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:34.600036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869d5af-c295-48f8-a30f-0697b72a6b65 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:34.600320 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869d5af-c295-48f8-a30f-0697b72a6b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:34.600596 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869d5af-c295-48f8-a30f-0697b72a6b65 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:34.601346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869d5af-c295-48f8-a30f-0697b72a6b65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:37:34.601657 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869d5af-c295-48f8-a30f-0697b72a6b65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:34.603444 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8869d5af-c295-48f8-a30f-0697b72a6b65 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:34.617019 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1158/2304] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:37:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:37:34.622887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869d5af-c295-48f8-a30f-0697b72a6b65 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:34.625646 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1147/2305] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:37:34 2022] POST /identity/v3/users => generated 293 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:34.629956 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd0cc05-9dcb-4af4-9f10-989aaac87faa admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:34.631799 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-afd0cc05-9dcb-4af4-9f10-989aaac87faa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:34.640973 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869d5af-c295-48f8-a30f-0697b72a6b65 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:34.645467 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd0cc05-9dcb-4af4-9f10-989aaac87faa admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:34.646747 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7c3c6-bc86-4723-adb8-b20eeeefbf2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:34.647490 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7c3c6-bc86-4723-adb8-b20eeeefbf2a admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:34.647773 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7c3c6-bc86-4723-adb8-b20eeeefbf2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:34.648083 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7c3c6-bc86-4723-adb8-b20eeeefbf2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:34.648745 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca7c3c6-bc86-4723-adb8-b20eeeefbf2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:34.650681 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca7c3c6-bc86-4723-adb8-b20eeeefbf2a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:34.662452 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27965acf-7fe1-476f-8339-d5581e3c64ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:34.663361 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27965acf-7fe1-476f-8339-d5581e3c64ec admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:34.663775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27965acf-7fe1-476f-8339-d5581e3c64ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:34.664287 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27965acf-7fe1-476f-8339-d5581e3c64ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:34.665211 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27965acf-7fe1-476f-8339-d5581e3c64ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:34.667350 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27965acf-7fe1-476f-8339-d5581e3c64ec admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:34.684175 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1148/2306] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:37:34.686585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27965acf-7fe1-476f-8339-d5581e3c64ec admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:34.700958 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27965acf-7fe1-476f-8339-d5581e3c64ec admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:34.710581 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50396a63-b2e3-47c2-83b8-8a0dfcf71861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:34.711483 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50396a63-b2e3-47c2-83b8-8a0dfcf71861 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:34.711768 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50396a63-b2e3-47c2-83b8-8a0dfcf71861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:34.712044 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50396a63-b2e3-47c2-83b8-8a0dfcf71861 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:34.712783 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50396a63-b2e3-47c2-83b8-8a0dfcf71861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:34.714450 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50396a63-b2e3-47c2-83b8-8a0dfcf71861 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:34.728432 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1149/2307] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:34.732864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50396a63-b2e3-47c2-83b8-8a0dfcf71861 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:34.736226 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:34.746615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50396a63-b2e3-47c2-83b8-8a0dfcf71861 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:34.751827 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f4d1a9-86a5-4883-a393-cd7408e3170e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:34.752800 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4d1a9-86a5-4883-a393-cd7408e3170e admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:34.753027 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4d1a9-86a5-4883-a393-cd7408e3170e admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:34.753304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4d1a9-86a5-4883-a393-cd7408e3170e admin admin] PATH_INFO: `/v3/projects/99d82d4c868f404e9bc7e932aadf7670/users/c81a63ee15d843bfb857a7f3148015e8/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:34.768824 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f4d1a9-86a5-4883-a393-cd7408e3170e admin admin] RBAC: Authorizing `identity:create_grant(project_id=99d82d4c868f404e9bc7e932aadf7670, user_id=c81a63ee15d843bfb857a7f3148015e8, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:34.772152 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f4d1a9-86a5-4883-a393-cd7408e3170e admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:34.822475 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1159/2308] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:37:34 2022] POST /identity/v3/projects => generated 389 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:34.826471 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7c3c6-bc86-4723-adb8-b20eeeefbf2a admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:34.891770 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7c3c6-bc86-4723-adb8-b20eeeefbf2a admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:34.896975 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c38089e-d5b9-4b20-91ac-4af8b952ab16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:34.930133 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38089e-d5b9-4b20-91ac-4af8b952ab16 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:34.930133 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38089e-d5b9-4b20-91ac-4af8b952ab16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:34.930133 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38089e-d5b9-4b20-91ac-4af8b952ab16 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:34.930133 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c38089e-d5b9-4b20-91ac-4af8b952ab16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:34.930133 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c38089e-d5b9-4b20-91ac-4af8b952ab16 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:35.244582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1150/2309] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:37:34 2022] PUT /identity/v3/projects/99d82d4c868f404e9bc7e932aadf7670/users/c81a63ee15d843bfb857a7f3148015e8/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:37:35.249631 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d869cdf-3b34-418e-bc87-ef7f47cdab05 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:35.253730 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d869cdf-3b34-418e-bc87-ef7f47cdab05 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:35.253730 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d869cdf-3b34-418e-bc87-ef7f47cdab05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:35.269553 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-5d869cdf-3b34-418e-bc87-ef7f47cdab05 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:35.271463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-5d869cdf-3b34-418e-bc87-ef7f47cdab05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:35.283638 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1160/2310] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:37:34 2022] POST /identity/v3/users => generated 295 bytes in 455 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:37:35.288228 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c38089e-d5b9-4b20-91ac-4af8b952ab16 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:35.296704 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c38089e-d5b9-4b20-91ac-4af8b952ab16 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:35.472227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1151/2311] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:35.482220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15690401-b41b-4e74-8001-0252f4ddfeee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:35.484911 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15690401-b41b-4e74-8001-0252f4ddfeee admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:35.487275 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15690401-b41b-4e74-8001-0252f4ddfeee admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:35.487368 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15690401-b41b-4e74-8001-0252f4ddfeee admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:35.500199 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15690401-b41b-4e74-8001-0252f4ddfeee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:35.500199 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15690401-b41b-4e74-8001-0252f4ddfeee admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:35.502606 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d869cdf-3b34-418e-bc87-ef7f47cdab05 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:35.512790 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-5d869cdf-3b34-418e-bc87-ef7f47cdab05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:35.527472 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1161/2312] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:37:35.528900 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15690401-b41b-4e74-8001-0252f4ddfeee admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:35.547561 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15690401-b41b-4e74-8001-0252f4ddfeee admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:35.560011 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea0570-06f2-42f0-a576-3d621fc25c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:35.560790 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea0570-06f2-42f0-a576-3d621fc25c20 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:35.561175 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea0570-06f2-42f0-a576-3d621fc25c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:35.561518 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea0570-06f2-42f0-a576-3d621fc25c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:35.562336 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea0570-06f2-42f0-a576-3d621fc25c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:35.564487 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea0570-06f2-42f0-a576-3d621fc25c20 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:35.589156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1162/2313] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:35.594698 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea0570-06f2-42f0-a576-3d621fc25c20 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:35.611892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea0570-06f2-42f0-a576-3d621fc25c20 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:35.623741 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372334a-6b78-4620-9286-96ac11255950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:35.625851 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372334a-6b78-4620-9286-96ac11255950 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:35.627231 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372334a-6b78-4620-9286-96ac11255950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:35.627523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8372334a-6b78-4620-9286-96ac11255950 admin admin] PATH_INFO: `/v3/projects/c712f8a35972479e93b29004cb7e59ab/users/7429ef02d8f44451a9977c5ce857d5be/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:35.631889 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d869cdf-3b34-418e-bc87-ef7f47cdab05 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:35.642932 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f290622-77ba-4de4-ac08-2db0bf93e1fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:35.646974 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f290622-77ba-4de4-ac08-2db0bf93e1fd service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:35.646974 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f290622-77ba-4de4-ac08-2db0bf93e1fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:35.646974 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f290622-77ba-4de4-ac08-2db0bf93e1fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:35.655592 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8372334a-6b78-4620-9286-96ac11255950 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c712f8a35972479e93b29004cb7e59ab, user_id=7429ef02d8f44451a9977c5ce857d5be, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:35.658424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8372334a-6b78-4620-9286-96ac11255950 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:35.670156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f290622-77ba-4de4-ac08-2db0bf93e1fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:35.672303 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f290622-77ba-4de4-ac08-2db0bf93e1fd service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:35.690604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1152/2314] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:37:35.795213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1163/2315] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:37:35 2022] PUT /identity/v3/projects/c712f8a35972479e93b29004cb7e59ab/users/7429ef02d8f44451a9977c5ce857d5be/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:35.798941 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401bde5-e607-4354-a7cc-cc8f499f8b09 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:35.799243 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401bde5-e607-4354-a7cc-cc8f499f8b09 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:35.799532 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1401bde5-e607-4354-a7cc-cc8f499f8b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:35.847058 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-1401bde5-e607-4354-a7cc-cc8f499f8b09 None None] MFA Rules not processed for user `7429ef02d8f44451a9977c5ce857d5be`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:35.850124 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1401bde5-e607-4354-a7cc-cc8f499f8b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:35.960019 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1153/2316] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:35.964738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372334a-6b78-4620-9286-96ac11255950 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:36.031533 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:36.087799 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8372334a-6b78-4620-9286-96ac11255950 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:36.093628 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee208e2c-b392-497b-9511-4ae6dd418456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:36.094757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee208e2c-b392-497b-9511-4ae6dd418456 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:36.095162 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee208e2c-b392-497b-9511-4ae6dd418456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:36.095534 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee208e2c-b392-497b-9511-4ae6dd418456 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:36.096359 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee208e2c-b392-497b-9511-4ae6dd418456 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:36.098803 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee208e2c-b392-497b-9511-4ae6dd418456 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:36.265085 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1164/2317] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:37:35 2022] POST /identity/v3/projects => generated 387 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:36.268929 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401bde5-e607-4354-a7cc-cc8f499f8b09 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:36.281255 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1401bde5-e607-4354-a7cc-cc8f499f8b09 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:36.362285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659a592e-881b-43fa-acd3-79c5172eb8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:36.363351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a592e-881b-43fa-acd3-79c5172eb8b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:36.363971 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a592e-881b-43fa-acd3-79c5172eb8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:36.364188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659a592e-881b-43fa-acd3-79c5172eb8b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:36.367553 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659a592e-881b-43fa-acd3-79c5172eb8b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:36.372219 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659a592e-881b-43fa-acd3-79c5172eb8b1 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:36.377426 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:36.491653 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1154/2318] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:37:36 2022] POST /identity/v3/users => generated 300 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:36.496058 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee208e2c-b392-497b-9511-4ae6dd418456 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:36.508584 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee208e2c-b392-497b-9511-4ae6dd418456 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:36.513946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d6eda-8c3e-4166-8d70-cf807e1987b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:36.515160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d6eda-8c3e-4166-8d70-cf807e1987b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:36.515451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d6eda-8c3e-4166-8d70-cf807e1987b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:36.515737 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d6eda-8c3e-4166-8d70-cf807e1987b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:36.517914 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d6eda-8c3e-4166-8d70-cf807e1987b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:36.519600 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d6eda-8c3e-4166-8d70-cf807e1987b3 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:36.535435 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1165/2319] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:36.537414 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659a592e-881b-43fa-acd3-79c5172eb8b1 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:36.566490 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659a592e-881b-43fa-acd3-79c5172eb8b1 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:36.576056 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a7517-6b04-47ed-9242-f94dbf67dea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:36.577641 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a7517-6b04-47ed-9242-f94dbf67dea1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:36.578241 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a7517-6b04-47ed-9242-f94dbf67dea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:36.578694 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a7517-6b04-47ed-9242-f94dbf67dea1 admin admin] PATH_INFO: `/v3/projects/a4a3cab85a114204801e715ba4901f4d/users/bc43125f3fed4b80bb439380fe16800a/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:36.610699 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4a7517-6b04-47ed-9242-f94dbf67dea1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4a3cab85a114204801e715ba4901f4d, user_id=bc43125f3fed4b80bb439380fe16800a, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:36.614669 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4a7517-6b04-47ed-9242-f94dbf67dea1 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:36.686892 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1155/2320] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:37:36 2022] PUT /identity/v3/projects/a4a3cab85a114204801e715ba4901f4d/users/bc43125f3fed4b80bb439380fe16800a/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:37:36.697045 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d6eda-8c3e-4166-8d70-cf807e1987b3 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:36.717764 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d6eda-8c3e-4166-8d70-cf807e1987b3 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:36.945631 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6e34c-8c2a-4572-a32a-deb4e5aa1c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:36.946222 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6e34c-8c2a-4572-a32a-deb4e5aa1c20 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:36.946510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6e34c-8c2a-4572-a32a-deb4e5aa1c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:36.946790 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6e34c-8c2a-4572-a32a-deb4e5aa1c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:36.947582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6e34c-8c2a-4572-a32a-deb4e5aa1c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:36.959254 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6e34c-8c2a-4572-a32a-deb4e5aa1c20 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:36.972392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1166/2321] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:36.976484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a7517-6b04-47ed-9242-f94dbf67dea1 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:36.990318 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a7517-6b04-47ed-9242-f94dbf67dea1 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:37.004038 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd62ec-d9fb-42d5-b8db-697f869d9026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:37.005181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd62ec-d9fb-42d5-b8db-697f869d9026 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:37.005569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd62ec-d9fb-42d5-b8db-697f869d9026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:37.006538 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd62ec-d9fb-42d5-b8db-697f869d9026 admin admin] PATH_INFO: `/v3/projects/a4a3cab85a114204801e715ba4901f4d/users/bc43125f3fed4b80bb439380fe16800a/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:37.013613 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd62ec-d9fb-42d5-b8db-697f869d9026 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4a3cab85a114204801e715ba4901f4d, user_id=bc43125f3fed4b80bb439380fe16800a, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:37.024898 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd62ec-d9fb-42d5-b8db-697f869d9026 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:37.061884 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1156/2322] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:37:36 2022] PUT /identity/v3/projects/a4a3cab85a114204801e715ba4901f4d/users/bc43125f3fed4b80bb439380fe16800a/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:37.063353 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421b3de-66d6-4b54-8c9f-624456104d76 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:37.063809 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421b3de-66d6-4b54-8c9f-624456104d76 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:37.064274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421b3de-66d6-4b54-8c9f-624456104d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:37.159594 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-1421b3de-66d6-4b54-8c9f-624456104d76 None None] MFA Rules not processed for user `bc43125f3fed4b80bb439380fe16800a`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:37.162765 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1421b3de-66d6-4b54-8c9f-624456104d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:37.227283 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:37.282702 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1167/2323] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:37:37.294042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd62ec-d9fb-42d5-b8db-697f869d9026 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:37.295942 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-fafd62ec-d9fb-42d5-b8db-697f869d9026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:37.363192 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd62ec-d9fb-42d5-b8db-697f869d9026 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:37.368530 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e5340-67d4-4935-ab34-51a0da377a68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:37.369455 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e5340-67d4-4935-ab34-51a0da377a68 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:37.369988 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e5340-67d4-4935-ab34-51a0da377a68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:37.370448 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e5340-67d4-4935-ab34-51a0da377a68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:37.382085 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e5340-67d4-4935-ab34-51a0da377a68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:37.383974 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e5340-67d4-4935-ab34-51a0da377a68 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:37.394901 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1157/2324] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:37.433020 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:37.562947 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421b3de-66d6-4b54-8c9f-624456104d76 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:37.640329 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421b3de-66d6-4b54-8c9f-624456104d76 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:37.648113 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac5885-d431-4907-b2c1-570a2fc7f99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:37.649233 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac5885-d431-4907-b2c1-570a2fc7f99d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:37.649724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac5885-d431-4907-b2c1-570a2fc7f99d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:37.650185 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ac5885-d431-4907-b2c1-570a2fc7f99d admin admin] PATH_INFO: `/v3/users/7429ef02d8f44451a9977c5ce857d5be` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:37.651928 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ac5885-d431-4907-b2c1-570a2fc7f99d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7429ef02d8f44451a9977c5ce857d5be)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:37.657510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ac5885-d431-4907-b2c1-570a2fc7f99d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:37.910184 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-05ac5885-d431-4907-b2c1-570a2fc7f99d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7429ef02d8f44451a9977c5ce857d5be'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:38.016975 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:38.017757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-05ac5885-d431-4907-b2c1-570a2fc7f99d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7429ef02d8f44451a9977c5ce857d5be'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:38.039777 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-05ac5885-d431-4907-b2c1-570a2fc7f99d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7429ef02d8f44451a9977c5ce857d5be'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:38.040529 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-05ac5885-d431-4907-b2c1-570a2fc7f99d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7429ef02d8f44451a9977c5ce857d5be'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:38.058352 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-05ac5885-d431-4907-b2c1-570a2fc7f99d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7429ef02d8f44451a9977c5ce857d5be'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:38.068424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1168/2325] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:37:37 2022] DELETE /identity/v3/users/7429ef02d8f44451a9977c5ce857d5be => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:38.072815 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8d81d-7472-4ec6-8e88-5ebf2d6f89b3 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:38.073694 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8d81d-7472-4ec6-8e88-5ebf2d6f89b3 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:38.078925 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8d81d-7472-4ec6-8e88-5ebf2d6f89b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:38.113629 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-25e8d81d-7472-4ec6-8e88-5ebf2d6f89b3 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:38.116288 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-25e8d81d-7472-4ec6-8e88-5ebf2d6f89b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:38.297385 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1158/2326] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:38.309127 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac5885-d431-4907-b2c1-570a2fc7f99d admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:38.310791 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-05ac5885-d431-4907-b2c1-570a2fc7f99d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:38.413017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ac5885-d431-4907-b2c1-570a2fc7f99d admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:38.417445 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66f84c-e523-4e69-b2e0-815355690226 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:38.421503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66f84c-e523-4e69-b2e0-815355690226 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:38.421503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66f84c-e523-4e69-b2e0-815355690226 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:38.421503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66f84c-e523-4e69-b2e0-815355690226 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:38.436194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e66f84c-e523-4e69-b2e0-815355690226 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:38.438049 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e66f84c-e523-4e69-b2e0-815355690226 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:38.449979 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1169/2327] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:37:38.653632 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:38.845371 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:39.318646 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:39.483200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8d81d-7472-4ec6-8e88-5ebf2d6f89b3 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:39.485276 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-25e8d81d-7472-4ec6-8e88-5ebf2d6f89b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:39.516067 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8d81d-7472-4ec6-8e88-5ebf2d6f89b3 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:39.522140 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7138c1d-dafe-43f2-a142-dcacffc6d6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:39.523527 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7138c1d-dafe-43f2-a142-dcacffc6d6aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:39.523527 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7138c1d-dafe-43f2-a142-dcacffc6d6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:39.524169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7138c1d-dafe-43f2-a142-dcacffc6d6aa admin admin] PATH_INFO: `/v3/projects/c712f8a35972479e93b29004cb7e59ab` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:39.525706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7138c1d-dafe-43f2-a142-dcacffc6d6aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=c712f8a35972479e93b29004cb7e59ab)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:39.527745 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7138c1d-dafe-43f2-a142-dcacffc6d6aa admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:39.704994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d7138c1d-dafe-43f2-a142-dcacffc6d6aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c712f8a35972479e93b29004cb7e59ab', 'request_id': 'req-d7138c1d-dafe-43f2-a142-dcacffc6d6aa'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:39.785005 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d7138c1d-dafe-43f2-a142-dcacffc6d6aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c712f8a35972479e93b29004cb7e59ab', 'request_id': 'req-d7138c1d-dafe-43f2-a142-dcacffc6d6aa'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:39.809230 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d7138c1d-dafe-43f2-a142-dcacffc6d6aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c712f8a35972479e93b29004cb7e59ab', 'request_id': 'req-d7138c1d-dafe-43f2-a142-dcacffc6d6aa'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:39.869549 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d7138c1d-dafe-43f2-a142-dcacffc6d6aa admin admin] The token cache is being invalidate because project c712f8a35972479e93b29004cb7e59ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:37:39.870015 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d7138c1d-dafe-43f2-a142-dcacffc6d6aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:39.870754 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-d7138c1d-dafe-43f2-a142-dcacffc6d6aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:39.876048 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1159/2328] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:37:39 2022] DELETE /identity/v3/projects/c712f8a35972479e93b29004cb7e59ab => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:39.877095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66f84c-e523-4e69-b2e0-815355690226 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:39.899748 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66f84c-e523-4e69-b2e0-815355690226 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:40.123588 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:40.142978 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888b202-1f59-46f2-89c4-8d638ab305e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:40.152153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888b202-1f59-46f2-89c4-8d638ab305e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:40.152752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888b202-1f59-46f2-89c4-8d638ab305e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:40.153538 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888b202-1f59-46f2-89c4-8d638ab305e9 admin admin] PATH_INFO: `/v3/users/bc43125f3fed4b80bb439380fe16800a` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:40.162799 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9888b202-1f59-46f2-89c4-8d638ab305e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc43125f3fed4b80bb439380fe16800a)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:40.165013 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9888b202-1f59-46f2-89c4-8d638ab305e9 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:40.785362 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:40.888698 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-9888b202-1f59-46f2-89c4-8d638ab305e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc43125f3fed4b80bb439380fe16800a'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:40.994893 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-9888b202-1f59-46f2-89c4-8d638ab305e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc43125f3fed4b80bb439380fe16800a'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:41.029877 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-9888b202-1f59-46f2-89c4-8d638ab305e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc43125f3fed4b80bb439380fe16800a'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:41.034659 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-9888b202-1f59-46f2-89c4-8d638ab305e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc43125f3fed4b80bb439380fe16800a'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:41.055752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-9888b202-1f59-46f2-89c4-8d638ab305e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc43125f3fed4b80bb439380fe16800a'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:41.095860 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1170/2329] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:37:39 2022] DELETE /identity/v3/users/bc43125f3fed4b80bb439380fe16800a => generated 0 bytes in 1219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:41.545211 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:42.283093 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:42.385593 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7138c1d-dafe-43f2-a142-dcacffc6d6aa admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:42.387694 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d7138c1d-dafe-43f2-a142-dcacffc6d6aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:42.412959 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7138c1d-dafe-43f2-a142-dcacffc6d6aa admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:42.509260 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f3544-8a43-4033-b6d9-f75311f1c8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:42.510166 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f3544-8a43-4033-b6d9-f75311f1c8b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:42.510451 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f3544-8a43-4033-b6d9-f75311f1c8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:42.510745 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f3544-8a43-4033-b6d9-f75311f1c8b1 admin admin] PATH_INFO: `/v3/projects/a4a3cab85a114204801e715ba4901f4d` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:42.511987 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f3544-8a43-4033-b6d9-f75311f1c8b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4a3cab85a114204801e715ba4901f4d)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:42.513913 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f3544-8a43-4033-b6d9-f75311f1c8b1 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:42.639520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ea1f3544-8a43-4033-b6d9-f75311f1c8b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4a3cab85a114204801e715ba4901f4d', 'request_id': 'req-ea1f3544-8a43-4033-b6d9-f75311f1c8b1'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:42.706930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ea1f3544-8a43-4033-b6d9-f75311f1c8b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4a3cab85a114204801e715ba4901f4d', 'request_id': 'req-ea1f3544-8a43-4033-b6d9-f75311f1c8b1'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:42.714474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ea1f3544-8a43-4033-b6d9-f75311f1c8b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4a3cab85a114204801e715ba4901f4d', 'request_id': 'req-ea1f3544-8a43-4033-b6d9-f75311f1c8b1'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:42.725447 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ea1f3544-8a43-4033-b6d9-f75311f1c8b1 admin admin] The token cache is being invalidate because project a4a3cab85a114204801e715ba4901f4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:37:42.725877 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ea1f3544-8a43-4033-b6d9-f75311f1c8b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:42.726544 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ea1f3544-8a43-4033-b6d9-f75311f1c8b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:42.728281 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1160/2330] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:37:42 2022] DELETE /identity/v3/projects/a4a3cab85a114204801e715ba4901f4d => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:42.733259 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a43ae-4e55-4045-93f4-62c4bbf58a83 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:42.733772 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a43ae-4e55-4045-93f4-62c4bbf58a83 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:42.734086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a43ae-4e55-4045-93f4-62c4bbf58a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:42.752491 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-596a43ae-4e55-4045-93f4-62c4bbf58a83 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:42.753697 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-596a43ae-4e55-4045-93f4-62c4bbf58a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:42.829593 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16138986-92e3-4fb5-8af8-fc6faefc36e5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:42.836278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16138986-92e3-4fb5-8af8-fc6faefc36e5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:42.836278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16138986-92e3-4fb5-8af8-fc6faefc36e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:42.836672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1171/2331] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:42.852388 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-16138986-92e3-4fb5-8af8-fc6faefc36e5 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:42.853601 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-16138986-92e3-4fb5-8af8-fc6faefc36e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:42.862366 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1161/2332] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:37:42.868388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a43ae-4e55-4045-93f4-62c4bbf58a83 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:42.882411 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a43ae-4e55-4045-93f4-62c4bbf58a83 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:42.890255 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600a40d-3f89-4c79-ae16-f86a69c72105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:42.891287 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600a40d-3f89-4c79-ae16-f86a69c72105 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:42.893757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600a40d-3f89-4c79-ae16-f86a69c72105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:42.893757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600a40d-3f89-4c79-ae16-f86a69c72105 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:42.893757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0600a40d-3f89-4c79-ae16-f86a69c72105 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:37:42.893757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0600a40d-3f89-4c79-ae16-f86a69c72105 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:42.894865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0600a40d-3f89-4c79-ae16-f86a69c72105 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:42.913322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1172/2333] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:37:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:37:42.917804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16138986-92e3-4fb5-8af8-fc6faefc36e5 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:42.933202 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16138986-92e3-4fb5-8af8-fc6faefc36e5 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:42.948105 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da8b8c-a6d5-420d-b6b0-b32e9e6325a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:42.951930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da8b8c-a6d5-420d-b6b0-b32e9e6325a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:42.958937 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da8b8c-a6d5-420d-b6b0-b32e9e6325a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:42.958937 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da8b8c-a6d5-420d-b6b0-b32e9e6325a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:42.958937 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da8b8c-a6d5-420d-b6b0-b32e9e6325a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:42.958937 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da8b8c-a6d5-420d-b6b0-b32e9e6325a7 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:43.096144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617aa33b-c1a5-4cd5-9cc4-24f2443a65f6 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:43.096438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617aa33b-c1a5-4cd5-9cc4-24f2443a65f6 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:43.096721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617aa33b-c1a5-4cd5-9cc4-24f2443a65f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:43.104065 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1162/2334] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:37:42 2022] POST /identity/v3/projects => generated 377 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:43.107850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da8b8c-a6d5-420d-b6b0-b32e9e6325a7 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:43.125129 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-617aa33b-c1a5-4cd5-9cc4-24f2443a65f6 None None] MFA Rules not processed for user `1d76f23d7b614fac857a9aefd2d345b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:43.126667 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-617aa33b-c1a5-4cd5-9cc4-24f2443a65f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:43.208774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1173/2335] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:43.272799 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da8b8c-a6d5-420d-b6b0-b32e9e6325a7 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:43.285152 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bcd25a-9083-4f4c-beb5-79224843c081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:43.285847 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcd25a-9083-4f4c-beb5-79224843c081 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:43.286206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcd25a-9083-4f4c-beb5-79224843c081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:43.286434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bcd25a-9083-4f4c-beb5-79224843c081 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:43.287169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bcd25a-9083-4f4c-beb5-79224843c081 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:43.289140 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bcd25a-9083-4f4c-beb5-79224843c081 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:43.465654 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1163/2336] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:37:43 2022] POST /identity/v3/users => generated 289 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:43.471058 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617aa33b-c1a5-4cd5-9cc4-24f2443a65f6 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:43.486539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617aa33b-c1a5-4cd5-9cc4-24f2443a65f6 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:43.505072 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ea60b-ff8c-48cd-93a0-f603cdf212f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:43.505769 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ea60b-ff8c-48cd-93a0-f603cdf212f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:43.506054 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ea60b-ff8c-48cd-93a0-f603cdf212f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:43.506334 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ea60b-ff8c-48cd-93a0-f603cdf212f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:43.507139 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6ea60b-ff8c-48cd-93a0-f603cdf212f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:43.509722 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6ea60b-ff8c-48cd-93a0-f603cdf212f5 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:43.525551 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1174/2337] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:43.528663 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bcd25a-9083-4f4c-beb5-79224843c081 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:43.541507 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bcd25a-9083-4f4c-beb5-79224843c081 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:43.546668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d622561-0fa5-456a-9372-65646a0f0f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:43.547372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d622561-0fa5-456a-9372-65646a0f0f31 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:43.547662 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d622561-0fa5-456a-9372-65646a0f0f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:43.548198 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d622561-0fa5-456a-9372-65646a0f0f31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:43.549102 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d622561-0fa5-456a-9372-65646a0f0f31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:43.550825 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d622561-0fa5-456a-9372-65646a0f0f31 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:43.561863 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1164/2338] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:43.566178 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ea60b-ff8c-48cd-93a0-f603cdf212f5 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:43.576830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ea60b-ff8c-48cd-93a0-f603cdf212f5 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:43.581878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a332047-5c8c-4371-a556-12840ad6bffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:43.582814 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a332047-5c8c-4371-a556-12840ad6bffe admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:43.583653 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a332047-5c8c-4371-a556-12840ad6bffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:43.583928 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a332047-5c8c-4371-a556-12840ad6bffe admin admin] PATH_INFO: `/v3/projects/5a58b2492ec44638af0a2f3f5638344d/users/681de29867574d2a944189e9b455b907/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:43.603102 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a332047-5c8c-4371-a556-12840ad6bffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a58b2492ec44638af0a2f3f5638344d, user_id=681de29867574d2a944189e9b455b907, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:43.606148 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a332047-5c8c-4371-a556-12840ad6bffe admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:43.697750 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1175/2339] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:37:43 2022] PUT /identity/v3/projects/5a58b2492ec44638af0a2f3f5638344d/users/681de29867574d2a944189e9b455b907/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:37:43.710225 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58f772-dfec-4a11-b2c7-a6230d15ab1b None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:43.710535 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58f772-dfec-4a11-b2c7-a6230d15ab1b None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:43.710965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58f772-dfec-4a11-b2c7-a6230d15ab1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:43.725797 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:43.754184 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-0d58f772-dfec-4a11-b2c7-a6230d15ab1b None None] MFA Rules not processed for user `681de29867574d2a944189e9b455b907`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:43.754742 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-0d58f772-dfec-4a11-b2c7-a6230d15ab1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:43.826709 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1165/2340] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:43.828227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a332047-5c8c-4371-a556-12840ad6bffe admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:43.915445 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a332047-5c8c-4371-a556-12840ad6bffe admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:43.923955 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d231f-c1ef-444c-a2ed-7f064ac169bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:43.924788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d231f-c1ef-444c-a2ed-7f064ac169bb admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:43.925080 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d231f-c1ef-444c-a2ed-7f064ac169bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:43.925419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d231f-c1ef-444c-a2ed-7f064ac169bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:43.937269 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0d231f-c1ef-444c-a2ed-7f064ac169bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:43.943135 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0d231f-c1ef-444c-a2ed-7f064ac169bb admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:44.092822 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1176/2341] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:37:43 2022] POST /identity/v3/projects => generated 377 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:44.097083 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58f772-dfec-4a11-b2c7-a6230d15ab1b None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:44.112500 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58f772-dfec-4a11-b2c7-a6230d15ab1b None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:44.205346 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2cb184-a41c-43a8-bc7d-ef440dc11928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:44.206033 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cb184-a41c-43a8-bc7d-ef440dc11928 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:44.206316 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cb184-a41c-43a8-bc7d-ef440dc11928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:44.206732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2cb184-a41c-43a8-bc7d-ef440dc11928 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:44.207700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2cb184-a41c-43a8-bc7d-ef440dc11928 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:44.210335 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2cb184-a41c-43a8-bc7d-ef440dc11928 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:44.318964 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1166/2342] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:37:44 2022] POST /identity/v3/users => generated 289 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:37:44.329117 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d231f-c1ef-444c-a2ed-7f064ac169bb admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:44.345013 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d231f-c1ef-444c-a2ed-7f064ac169bb admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:44.377279 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733d849-7c93-417d-b091-10e8a8b7dee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:44.383820 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733d849-7c93-417d-b091-10e8a8b7dee6 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:44.384114 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733d849-7c93-417d-b091-10e8a8b7dee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:44.384394 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733d849-7c93-417d-b091-10e8a8b7dee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:44.385138 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5733d849-7c93-417d-b091-10e8a8b7dee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:44.386781 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5733d849-7c93-417d-b091-10e8a8b7dee6 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:44.417171 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1177/2343] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:44.420876 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2cb184-a41c-43a8-bc7d-ef440dc11928 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:44.444975 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2cb184-a41c-43a8-bc7d-ef440dc11928 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:44.469538 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948aa34-d537-47a0-898f-d055fc056084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:44.473127 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948aa34-d537-47a0-898f-d055fc056084 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:44.476616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948aa34-d537-47a0-898f-d055fc056084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:44.477234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948aa34-d537-47a0-898f-d055fc056084 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:44.478155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948aa34-d537-47a0-898f-d055fc056084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:44.487193 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948aa34-d537-47a0-898f-d055fc056084 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:44.511752 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1167/2344] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:44 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:44.515575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733d849-7c93-417d-b091-10e8a8b7dee6 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:44.532248 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733d849-7c93-417d-b091-10e8a8b7dee6 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:44.543981 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17b2a3-feca-41c6-b060-98eaacf790c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:44.545549 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17b2a3-feca-41c6-b060-98eaacf790c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:44.545839 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17b2a3-feca-41c6-b060-98eaacf790c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:44.546117 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd17b2a3-feca-41c6-b060-98eaacf790c4 admin admin] PATH_INFO: `/v3/projects/6cb8f51ff63043e0b6dd6720c4ef4ee1/users/33f22a37735f4f9a999ed90868e939d7/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:44.584395 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd17b2a3-feca-41c6-b060-98eaacf790c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cb8f51ff63043e0b6dd6720c4ef4ee1, user_id=33f22a37735f4f9a999ed90868e939d7, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:44.587230 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd17b2a3-feca-41c6-b060-98eaacf790c4 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:44.687435 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1178/2345] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:37:44 2022] PUT /identity/v3/projects/6cb8f51ff63043e0b6dd6720c4ef4ee1/users/33f22a37735f4f9a999ed90868e939d7/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:44.689341 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a81cf-d1fc-4695-8db8-44353f346ac5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:44.691016 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a81cf-d1fc-4695-8db8-44353f346ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:44.693276 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2a81cf-d1fc-4695-8db8-44353f346ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:44.743517 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-0c2a81cf-d1fc-4695-8db8-44353f346ac5 None None] MFA Rules not processed for user `33f22a37735f4f9a999ed90868e939d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:44.751712 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-0c2a81cf-d1fc-4695-8db8-44353f346ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:44.773389 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:44.856066 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1168/2346] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:44.860493 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17b2a3-feca-41c6-b060-98eaacf790c4 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:44.945092 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd17b2a3-feca-41c6-b060-98eaacf790c4 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:44.950758 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825d19b-8441-4474-a8bf-ef397c4d66f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:44.957409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825d19b-8441-4474-a8bf-ef397c4d66f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:44.957409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825d19b-8441-4474-a8bf-ef397c4d66f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:44.957409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6825d19b-8441-4474-a8bf-ef397c4d66f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:44.957409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6825d19b-8441-4474-a8bf-ef397c4d66f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:44.957409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6825d19b-8441-4474-a8bf-ef397c4d66f4 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:45.100892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1179/2347] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:37:44 2022] POST /identity/v3/projects => generated 379 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:45.106563 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a81cf-d1fc-4695-8db8-44353f346ac5 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:45.118287 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2a81cf-d1fc-4695-8db8-44353f346ac5 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:45.191593 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84cb0d2-ae1f-415a-ad38-aa1181b41bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:45.192562 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84cb0d2-ae1f-415a-ad38-aa1181b41bfc admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:45.192996 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84cb0d2-ae1f-415a-ad38-aa1181b41bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:45.193437 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84cb0d2-ae1f-415a-ad38-aa1181b41bfc admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:45.194393 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84cb0d2-ae1f-415a-ad38-aa1181b41bfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:45.196756 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84cb0d2-ae1f-415a-ad38-aa1181b41bfc admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:45.315741 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825d19b-8441-4474-a8bf-ef397c4d66f4 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:45.323184 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1169/2348] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:37:45 2022] POST /identity/v3/users => generated 296 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:45.327864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6825d19b-8441-4474-a8bf-ef397c4d66f4 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:45.334427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f3a41-6666-4b45-a12e-52e5fb12a683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:45.340058 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f3a41-6666-4b45-a12e-52e5fb12a683 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:45.340058 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f3a41-6666-4b45-a12e-52e5fb12a683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:45.340058 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f3a41-6666-4b45-a12e-52e5fb12a683 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:45.340058 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f3a41-6666-4b45-a12e-52e5fb12a683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:45.340058 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f3a41-6666-4b45-a12e-52e5fb12a683 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:45.358049 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84cb0d2-ae1f-415a-ad38-aa1181b41bfc admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:45.375034 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1180/2349] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:45.382459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84cb0d2-ae1f-415a-ad38-aa1181b41bfc admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:45.393033 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:45.398207 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfff056-a8ae-4474-9834-f2884d8f5608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:45.399056 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfff056-a8ae-4474-9834-f2884d8f5608 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:45.399338 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfff056-a8ae-4474-9834-f2884d8f5608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:45.399610 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfff056-a8ae-4474-9834-f2884d8f5608 admin admin] PATH_INFO: `/v3/projects/2e17971f56cb4be4bd29fcf253216d04/users/784bfb7ab0d8432a84ac52eec2cf58c0/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:45.447222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfff056-a8ae-4474-9834-f2884d8f5608 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e17971f56cb4be4bd29fcf253216d04, user_id=784bfb7ab0d8432a84ac52eec2cf58c0, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:45.451911 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfff056-a8ae-4474-9834-f2884d8f5608 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:45.531291 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1170/2350] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:37:45 2022] PUT /identity/v3/projects/2e17971f56cb4be4bd29fcf253216d04/users/784bfb7ab0d8432a84ac52eec2cf58c0/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:45.532038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f3a41-6666-4b45-a12e-52e5fb12a683 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:45.542397 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f3a41-6666-4b45-a12e-52e5fb12a683 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:45.602672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff642b70-278c-4f8d-9e2b-3ed69f17fc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:45.603351 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff642b70-278c-4f8d-9e2b-3ed69f17fc02 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:45.613065 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff642b70-278c-4f8d-9e2b-3ed69f17fc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:45.613065 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff642b70-278c-4f8d-9e2b-3ed69f17fc02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:45.613065 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff642b70-278c-4f8d-9e2b-3ed69f17fc02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:45.613065 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff642b70-278c-4f8d-9e2b-3ed69f17fc02 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:45.618881 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1181/2351] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:37:45.622006 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfff056-a8ae-4474-9834-f2884d8f5608 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:45.634102 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfff056-a8ae-4474-9834-f2884d8f5608 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:45.640893 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701dee65-57d9-44c3-a8af-83adc224cc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:45.642076 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701dee65-57d9-44c3-a8af-83adc224cc6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:45.642437 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701dee65-57d9-44c3-a8af-83adc224cc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:45.642776 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701dee65-57d9-44c3-a8af-83adc224cc6a admin admin] PATH_INFO: `/v3/projects/2e17971f56cb4be4bd29fcf253216d04/users/784bfb7ab0d8432a84ac52eec2cf58c0/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:45.645056 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701dee65-57d9-44c3-a8af-83adc224cc6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e17971f56cb4be4bd29fcf253216d04, user_id=784bfb7ab0d8432a84ac52eec2cf58c0, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:45.648432 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701dee65-57d9-44c3-a8af-83adc224cc6a admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:45.663051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1171/2352] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:37:45 2022] PUT /identity/v3/projects/2e17971f56cb4be4bd29fcf253216d04/users/784bfb7ab0d8432a84ac52eec2cf58c0/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:45.679946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ae2fc-2813-4e9b-b207-c85cb468c72a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:45.680484 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ae2fc-2813-4e9b-b207-c85cb468c72a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:45.680870 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ae2fc-2813-4e9b-b207-c85cb468c72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:45.713484 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-815ae2fc-2813-4e9b-b207-c85cb468c72a None None] MFA Rules not processed for user `784bfb7ab0d8432a84ac52eec2cf58c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:45.714799 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-815ae2fc-2813-4e9b-b207-c85cb468c72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:45.815876 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1182/2353] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:45 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:45.829225 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701dee65-57d9-44c3-a8af-83adc224cc6a admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:45.831092 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-701dee65-57d9-44c3-a8af-83adc224cc6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:45.900279 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701dee65-57d9-44c3-a8af-83adc224cc6a admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:45.904334 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046666b4-c7b6-4e5e-92c1-6bb763150600 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:45.904992 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046666b4-c7b6-4e5e-92c1-6bb763150600 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:45.905277 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046666b4-c7b6-4e5e-92c1-6bb763150600 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:45.905580 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046666b4-c7b6-4e5e-92c1-6bb763150600 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:45.917013 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046666b4-c7b6-4e5e-92c1-6bb763150600 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:45.918641 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046666b4-c7b6-4e5e-92c1-6bb763150600 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:45.931488 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1172/2354] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:45 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:46.036354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ae2fc-2813-4e9b-b207-c85cb468c72a None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:46.037827 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-815ae2fc-2813-4e9b-b207-c85cb468c72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:46.050861 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ae2fc-2813-4e9b-b207-c85cb468c72a None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:46.058630 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783bf38e-b695-4864-9b4c-b2a378e96f54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:46.059393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783bf38e-b695-4864-9b4c-b2a378e96f54 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:46.061019 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783bf38e-b695-4864-9b4c-b2a378e96f54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:46.061019 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783bf38e-b695-4864-9b4c-b2a378e96f54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:46.079813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524d2e9-b0ab-422a-a17e-093abbe681b9 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:46.080095 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524d2e9-b0ab-422a-a17e-093abbe681b9 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:46.080749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524d2e9-b0ab-422a-a17e-093abbe681b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:46.109775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:46.115758 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d524d2e9-b0ab-422a-a17e-093abbe681b9 None None] MFA Rules not processed for user `c81a63ee15d843bfb857a7f3148015e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:46.116995 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d524d2e9-b0ab-422a-a17e-093abbe681b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:46.152867 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783bf38e-b695-4864-9b4c-b2a378e96f54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:46.154431 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783bf38e-b695-4864-9b4c-b2a378e96f54 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:46.166090 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1183/2355] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:46.197488 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1173/2356] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:46.202548 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783bf38e-b695-4864-9b4c-b2a378e96f54 service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:46.206975 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-783bf38e-b695-4864-9b4c-b2a378e96f54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:46.279002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783bf38e-b695-4864-9b4c-b2a378e96f54 service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:46.284804 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c36e4-9e77-49a5-bb65-e8bf07f5cbca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:46.285940 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c36e4-9e77-49a5-bb65-e8bf07f5cbca admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:46.286997 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c36e4-9e77-49a5-bb65-e8bf07f5cbca admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:46.287501 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c36e4-9e77-49a5-bb65-e8bf07f5cbca admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:46.288855 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42c36e4-9e77-49a5-bb65-e8bf07f5cbca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:46.290772 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42c36e4-9e77-49a5-bb65-e8bf07f5cbca admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:46.421872 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1184/2357] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:37:46 2022] POST /identity/v3/projects => generated 385 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:37:46.423001 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524d2e9-b0ab-422a-a17e-093abbe681b9 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:46.439877 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d524d2e9-b0ab-422a-a17e-093abbe681b9 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:46.541971 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0faef0-ccbd-48ab-8892-d157ddc90da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:46.542761 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0faef0-ccbd-48ab-8892-d157ddc90da1 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:46.543262 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0faef0-ccbd-48ab-8892-d157ddc90da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:46.543651 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0faef0-ccbd-48ab-8892-d157ddc90da1 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:46.544434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0faef0-ccbd-48ab-8892-d157ddc90da1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:46.547194 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0faef0-ccbd-48ab-8892-d157ddc90da1 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:46.557444 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:46.709823 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1174/2358] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:37:46 2022] POST /identity/v3/users => generated 299 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:37:46.713899 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c36e4-9e77-49a5-bb65-e8bf07f5cbca admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:46.729537 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c36e4-9e77-49a5-bb65-e8bf07f5cbca admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:46.733016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:46.736864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d2f1d-bfa9-4913-bd53-3fdc96957e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:46.737677 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d2f1d-bfa9-4913-bd53-3fdc96957e59 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:46.738141 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d2f1d-bfa9-4913-bd53-3fdc96957e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:46.738496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d2f1d-bfa9-4913-bd53-3fdc96957e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:46.739425 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43d2f1d-bfa9-4913-bd53-3fdc96957e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:46.741281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43d2f1d-bfa9-4913-bd53-3fdc96957e59 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:46.758308 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1185/2359] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:46.766464 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0faef0-ccbd-48ab-8892-d157ddc90da1 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:46.781143 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0faef0-ccbd-48ab-8892-d157ddc90da1 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:46.790377 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a53ee-8fd3-4f06-939f-4fc8b51e6d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:46.791631 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a53ee-8fd3-4f06-939f-4fc8b51e6d36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:46.791939 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a53ee-8fd3-4f06-939f-4fc8b51e6d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:46.792291 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a53ee-8fd3-4f06-939f-4fc8b51e6d36 admin admin] PATH_INFO: `/v3/projects/f19d9f99f9b94e21b1ce3676c570f24b/users/75e6fa4efa97411b885f1f7b0add735a/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:46.813553 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a53ee-8fd3-4f06-939f-4fc8b51e6d36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f19d9f99f9b94e21b1ce3676c570f24b, user_id=75e6fa4efa97411b885f1f7b0add735a, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:46.819318 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a53ee-8fd3-4f06-939f-4fc8b51e6d36 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:46.909475 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1175/2360] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:37:46 2022] PUT /identity/v3/projects/f19d9f99f9b94e21b1ce3676c570f24b/users/75e6fa4efa97411b885f1f7b0add735a/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:46.912446 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d2f1d-bfa9-4913-bd53-3fdc96957e59 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:46.939746 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d2f1d-bfa9-4913-bd53-3fdc96957e59 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:47.017055 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da116709-942c-4cc4-8870-44b494acd102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:47.017964 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da116709-942c-4cc4-8870-44b494acd102 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:47.028523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da116709-942c-4cc4-8870-44b494acd102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:47.028523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da116709-942c-4cc4-8870-44b494acd102 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:47.028523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da116709-942c-4cc4-8870-44b494acd102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:47.028523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da116709-942c-4cc4-8870-44b494acd102 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:47.035689 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1186/2361] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:37:47.040114 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a53ee-8fd3-4f06-939f-4fc8b51e6d36 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:47.052550 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a53ee-8fd3-4f06-939f-4fc8b51e6d36 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:47.058600 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c4944-beba-48c2-9f06-51b01170fec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:47.059277 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c4944-beba-48c2-9f06-51b01170fec6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:47.059587 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c4944-beba-48c2-9f06-51b01170fec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:47.059856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c4944-beba-48c2-9f06-51b01170fec6 admin admin] PATH_INFO: `/v3/projects/f19d9f99f9b94e21b1ce3676c570f24b/users/75e6fa4efa97411b885f1f7b0add735a/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:47.062904 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c4944-beba-48c2-9f06-51b01170fec6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f19d9f99f9b94e21b1ce3676c570f24b, user_id=75e6fa4efa97411b885f1f7b0add735a, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:47.066627 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c4944-beba-48c2-9f06-51b01170fec6 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:47.088185 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1176/2362] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:37:47 2022] PUT /identity/v3/projects/f19d9f99f9b94e21b1ce3676c570f24b/users/75e6fa4efa97411b885f1f7b0add735a/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:37:47.980793 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:48.231296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:49.718717 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:49.873266 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:51.031424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da116709-942c-4cc4-8870-44b494acd102 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:51.044247 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da116709-942c-4cc4-8870-44b494acd102 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:51.149657 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f53134-1028-4cc8-a88b-d8be2185a5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:51.150949 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f53134-1028-4cc8-a88b-d8be2185a5a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:51.151423 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f53134-1028-4cc8-a88b-d8be2185a5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:51.151857 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f53134-1028-4cc8-a88b-d8be2185a5a3 admin admin] PATH_INFO: `/v3/users/681de29867574d2a944189e9b455b907` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:51.157617 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f53134-1028-4cc8-a88b-d8be2185a5a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=681de29867574d2a944189e9b455b907)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:51.159906 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f53134-1028-4cc8-a88b-d8be2185a5a3 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:51.311059 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:51.401051 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-93f53134-1028-4cc8-a88b-d8be2185a5a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '681de29867574d2a944189e9b455b907'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:51.533179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-93f53134-1028-4cc8-a88b-d8be2185a5a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '681de29867574d2a944189e9b455b907'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:51.549951 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-93f53134-1028-4cc8-a88b-d8be2185a5a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '681de29867574d2a944189e9b455b907'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:51.550699 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-93f53134-1028-4cc8-a88b-d8be2185a5a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '681de29867574d2a944189e9b455b907'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:51.567959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-93f53134-1028-4cc8-a88b-d8be2185a5a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '681de29867574d2a944189e9b455b907'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:51.571717 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:51.578916 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1187/2363] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:37:51 2022] DELETE /identity/v3/users/681de29867574d2a944189e9b455b907 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:51.583309 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552095a6-fca2-48f7-882d-011300b0708b None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:51.583797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552095a6-fca2-48f7-882d-011300b0708b None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:51.584135 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552095a6-fca2-48f7-882d-011300b0708b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:51.604333 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-552095a6-fca2-48f7-882d-011300b0708b None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:51.605698 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-552095a6-fca2-48f7-882d-011300b0708b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:51.682910 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1177/2364] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:51.694507 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f53134-1028-4cc8-a88b-d8be2185a5a3 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:51.696486 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-93f53134-1028-4cc8-a88b-d8be2185a5a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:51.779086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f53134-1028-4cc8-a88b-d8be2185a5a3 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:51.784866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2601fd32-8975-4439-8477-c2c70ee4e888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:51.785234 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295537ad-b5c3-493b-b8b3-be7731a01076 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:51.785617 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601fd32-8975-4439-8477-c2c70ee4e888 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:51.785757 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295537ad-b5c3-493b-b8b3-be7731a01076 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:51.785906 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601fd32-8975-4439-8477-c2c70ee4e888 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:51.786334 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295537ad-b5c3-493b-b8b3-be7731a01076 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:51.786451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601fd32-8975-4439-8477-c2c70ee4e888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:51.788023 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1178/2365] 10.222.0.24 () {54 vars in 921 bytes} [Fri May 27 03:37:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:37:51.795415 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8566c4-e7c3-429e-a8cf-7b5250c7bcf0 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:51.798671 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8566c4-e7c3-429e-a8cf-7b5250c7bcf0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:51.798671 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8566c4-e7c3-429e-a8cf-7b5250c7bcf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:51.805167 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2601fd32-8975-4439-8477-c2c70ee4e888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:51.806850 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2601fd32-8975-4439-8477-c2c70ee4e888 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:51.817994 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1188/2366] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:51.895085 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-bf8566c4-e7c3-429e-a8cf-7b5250c7bcf0 None None] MFA Rules not processed for user `22a9592143014f3287cd75c60ca17237`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:51.897587 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-bf8566c4-e7c3-429e-a8cf-7b5250c7bcf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:51.994033 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1179/2367] 10.222.0.24 () {58 vars in 1018 bytes} [Fri May 27 03:37:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:37:52.013085 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2601fd32-8975-4439-8477-c2c70ee4e888 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:52.014544 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-2601fd32-8975-4439-8477-c2c70ee4e888 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:52.095401 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:52.109723 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2601fd32-8975-4439-8477-c2c70ee4e888 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:52.114005 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d27000-7b78-45e5-b704-435895fec426 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95452d1b892a463d8e9e3eb16ca7c5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:52.115568 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d27000-7b78-45e5-b704-435895fec426 service glance] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:52.116011 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d27000-7b78-45e5-b704-435895fec426 service glance] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:52.116438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d27000-7b78-45e5-b704-435895fec426 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:52.129269 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d27000-7b78-45e5-b704-435895fec426 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:52.131112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d27000-7b78-45e5-b704-435895fec426 service glance] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:52.146179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1189/2368] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:52.400845 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2ebc9-a05b-416a-8117-deda70ccd044 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:52.401820 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2ebc9-a05b-416a-8117-deda70ccd044 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:52.401820 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2ebc9-a05b-416a-8117-deda70ccd044 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:52.403052 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1180/2369] 10.222.0.24 () {54 vars in 921 bytes} [Fri May 27 03:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:37:52.415565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33d83a-b63b-4bcb-8fa4-4a7114309e72 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:52.416190 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33d83a-b63b-4bcb-8fa4-4a7114309e72 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:52.416552 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33d83a-b63b-4bcb-8fa4-4a7114309e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:52.459750 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-4f33d83a-b63b-4bcb-8fa4-4a7114309e72 None None] MFA Rules not processed for user `1af0160cd3da4a0bb391b81f083715d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:52.465595 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4f33d83a-b63b-4bcb-8fa4-4a7114309e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:52.493284 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1190/2370] 10.222.0.24 () {58 vars in 1018 bytes} [Fri May 27 03:37:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:52.518579 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b2ebc9-a05b-416a-8117-deda70ccd044 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:52.523358 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:52.525706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-13b2ebc9-a05b-416a-8117-deda70ccd044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:52.557636 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b2ebc9-a05b-416a-8117-deda70ccd044 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:52.562224 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57808110-c7bb-488e-a3d7-b15138f56c1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:52.563436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57808110-c7bb-488e-a3d7-b15138f56c1c service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:52.563953 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57808110-c7bb-488e-a3d7-b15138f56c1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:52.564409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57808110-c7bb-488e-a3d7-b15138f56c1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:52.576665 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57808110-c7bb-488e-a3d7-b15138f56c1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:52.584342 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57808110-c7bb-488e-a3d7-b15138f56c1c service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:52.591124 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33d83a-b63b-4bcb-8fa4-4a7114309e72 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:52.593004 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4f33d83a-b63b-4bcb-8fa4-4a7114309e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:52.606220 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1181/2371] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:52.620903 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f33d83a-b63b-4bcb-8fa4-4a7114309e72 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:52.627105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669cee94-aa49-4c87-a04a-e4dfb9fc6ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:52.628888 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669cee94-aa49-4c87-a04a-e4dfb9fc6ea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:52.629406 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669cee94-aa49-4c87-a04a-e4dfb9fc6ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:52.630448 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669cee94-aa49-4c87-a04a-e4dfb9fc6ea7 admin admin] PATH_INFO: `/v3/projects/5a58b2492ec44638af0a2f3f5638344d` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:52.631908 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669cee94-aa49-4c87-a04a-e4dfb9fc6ea7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a58b2492ec44638af0a2f3f5638344d)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:52.641496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669cee94-aa49-4c87-a04a-e4dfb9fc6ea7 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:52.827682 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-669cee94-aa49-4c87-a04a-e4dfb9fc6ea7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a58b2492ec44638af0a2f3f5638344d', 'request_id': 'req-669cee94-aa49-4c87-a04a-e4dfb9fc6ea7'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:52.834490 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7dba8-ee79-4344-8902-1faa4d450bd8 None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:52.834782 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7dba8-ee79-4344-8902-1faa4d450bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:52.835089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7dba8-ee79-4344-8902-1faa4d450bd8 None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:52.836476 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1182/2372] 10.222.0.24 () {54 vars in 921 bytes} [Fri May 27 03:37:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:37:52.844661 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a86b36-8034-4bbe-a94e-fdab23a4d9ce None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:52.845031 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a86b36-8034-4bbe-a94e-fdab23a4d9ce None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:52.845440 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a86b36-8034-4bbe-a94e-fdab23a4d9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:52.871807 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-44a86b36-8034-4bbe-a94e-fdab23a4d9ce None None] MFA Rules not processed for user `22a9592143014f3287cd75c60ca17237`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:52.876187 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-44a86b36-8034-4bbe-a94e-fdab23a4d9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:52.908263 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-669cee94-aa49-4c87-a04a-e4dfb9fc6ea7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a58b2492ec44638af0a2f3f5638344d', 'request_id': 'req-669cee94-aa49-4c87-a04a-e4dfb9fc6ea7'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:52.915366 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-669cee94-aa49-4c87-a04a-e4dfb9fc6ea7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a58b2492ec44638af0a2f3f5638344d', 'request_id': 'req-669cee94-aa49-4c87-a04a-e4dfb9fc6ea7'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:52.938389 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-669cee94-aa49-4c87-a04a-e4dfb9fc6ea7 admin admin] The token cache is being invalidate because project 5a58b2492ec44638af0a2f3f5638344d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:37:52.942185 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-669cee94-aa49-4c87-a04a-e4dfb9fc6ea7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:52.944274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-669cee94-aa49-4c87-a04a-e4dfb9fc6ea7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:52.946879 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1191/2373] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:37:52 2022] DELETE /identity/v3/projects/5a58b2492ec44638af0a2f3f5638344d => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:52.950754 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669cee94-aa49-4c87-a04a-e4dfb9fc6ea7 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:52.954106 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-669cee94-aa49-4c87-a04a-e4dfb9fc6ea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:52.967770 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1183/2374] 10.222.0.24 () {58 vars in 1018 bytes} [Fri May 27 03:37:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:37:52.984053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a86b36-8034-4bbe-a94e-fdab23a4d9ce None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:52.987918 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-44a86b36-8034-4bbe-a94e-fdab23a4d9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:53.113185 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669cee94-aa49-4c87-a04a-e4dfb9fc6ea7 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:53.122528 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a86b36-8034-4bbe-a94e-fdab23a4d9ce None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:53.125855 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7c215-89b2-4917-8588-bf70b03b5044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:53.126740 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7c215-89b2-4917-8588-bf70b03b5044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:53.127083 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7c215-89b2-4917-8588-bf70b03b5044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:53.127303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7c215-89b2-4917-8588-bf70b03b5044 admin admin] PATH_INFO: `/v3/users/33f22a37735f4f9a999ed90868e939d7` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:53.129243 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7c215-89b2-4917-8588-bf70b03b5044 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33f22a37735f4f9a999ed90868e939d7)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:53.132280 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7c215-89b2-4917-8588-bf70b03b5044 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:53.133925 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e61d15-6283-458b-9027-fe4bad1f25d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:53.134958 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e61d15-6283-458b-9027-fe4bad1f25d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:53.135386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e61d15-6283-458b-9027-fe4bad1f25d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:53.135835 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e61d15-6283-458b-9027-fe4bad1f25d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:53.149911 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e61d15-6283-458b-9027-fe4bad1f25d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:53.151413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e61d15-6283-458b-9027-fe4bad1f25d0 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:53.163809 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1184/2375] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:53.345945 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-31c7c215-89b2-4917-8588-bf70b03b5044 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33f22a37735f4f9a999ed90868e939d7'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:53.446088 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-31c7c215-89b2-4917-8588-bf70b03b5044 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33f22a37735f4f9a999ed90868e939d7'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:53.485552 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-31c7c215-89b2-4917-8588-bf70b03b5044 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33f22a37735f4f9a999ed90868e939d7'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:53.486338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-31c7c215-89b2-4917-8588-bf70b03b5044 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33f22a37735f4f9a999ed90868e939d7'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:53.500184 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-31c7c215-89b2-4917-8588-bf70b03b5044 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33f22a37735f4f9a999ed90868e939d7'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:53.508403 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1192/2376] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:37:52 2022] DELETE /identity/v3/users/33f22a37735f4f9a999ed90868e939d7 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:37:54.215542 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:54.894374 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:54.997031 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e61d15-6283-458b-9027-fe4bad1f25d0 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:55.011919 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e61d15-6283-458b-9027-fe4bad1f25d0 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:55.101235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464fe83-04d1-4de7-a107-4fd475c9e6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:55.102504 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464fe83-04d1-4de7-a107-4fd475c9e6ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:55.103074 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464fe83-04d1-4de7-a107-4fd475c9e6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:55.103591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464fe83-04d1-4de7-a107-4fd475c9e6ce admin admin] PATH_INFO: `/v3/projects/6cb8f51ff63043e0b6dd6720c4ef4ee1` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:55.105573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464fe83-04d1-4de7-a107-4fd475c9e6ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cb8f51ff63043e0b6dd6720c4ef4ee1)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:55.120677 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464fe83-04d1-4de7-a107-4fd475c9e6ce admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:55.343313 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c464fe83-04d1-4de7-a107-4fd475c9e6ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cb8f51ff63043e0b6dd6720c4ef4ee1', 'request_id': 'req-c464fe83-04d1-4de7-a107-4fd475c9e6ce'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:55.450082 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c464fe83-04d1-4de7-a107-4fd475c9e6ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cb8f51ff63043e0b6dd6720c4ef4ee1', 'request_id': 'req-c464fe83-04d1-4de7-a107-4fd475c9e6ce'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:55.459962 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c464fe83-04d1-4de7-a107-4fd475c9e6ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cb8f51ff63043e0b6dd6720c4ef4ee1', 'request_id': 'req-c464fe83-04d1-4de7-a107-4fd475c9e6ce'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:55.478606 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c464fe83-04d1-4de7-a107-4fd475c9e6ce admin admin] The token cache is being invalidate because project 6cb8f51ff63043e0b6dd6720c4ef4ee1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:37:55.478928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c464fe83-04d1-4de7-a107-4fd475c9e6ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:55.485048 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c464fe83-04d1-4de7-a107-4fd475c9e6ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:55.486830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1185/2377] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:37:54 2022] DELETE /identity/v3/projects/6cb8f51ff63043e0b6dd6720c4ef4ee1 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:55.491338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7c215-89b2-4917-8588-bf70b03b5044 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:55.492856 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-31c7c215-89b2-4917-8588-bf70b03b5044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:55.507733 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7c215-89b2-4917-8588-bf70b03b5044 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:55.580147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50687394-0e3e-465d-a89b-bafe3eedc9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:55.580147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50687394-0e3e-465d-a89b-bafe3eedc9f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:55.580147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50687394-0e3e-465d-a89b-bafe3eedc9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:55.580147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50687394-0e3e-465d-a89b-bafe3eedc9f0 admin admin] PATH_INFO: `/v3/users/784bfb7ab0d8432a84ac52eec2cf58c0` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:55.581006 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50687394-0e3e-465d-a89b-bafe3eedc9f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=784bfb7ab0d8432a84ac52eec2cf58c0)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:55.581006 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50687394-0e3e-465d-a89b-bafe3eedc9f0 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:55.833374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-50687394-0e3e-465d-a89b-bafe3eedc9f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '784bfb7ab0d8432a84ac52eec2cf58c0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:55.901641 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-50687394-0e3e-465d-a89b-bafe3eedc9f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '784bfb7ab0d8432a84ac52eec2cf58c0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:55.916472 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-50687394-0e3e-465d-a89b-bafe3eedc9f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '784bfb7ab0d8432a84ac52eec2cf58c0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:55.917410 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-50687394-0e3e-465d-a89b-bafe3eedc9f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '784bfb7ab0d8432a84ac52eec2cf58c0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:55.926106 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-50687394-0e3e-465d-a89b-bafe3eedc9f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '784bfb7ab0d8432a84ac52eec2cf58c0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:55.936589 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1193/2378] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:37:55 2022] DELETE /identity/v3/users/784bfb7ab0d8432a84ac52eec2cf58c0 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:37:56.645631 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa225a40-4a5c-4000-9b56-abb09f697934 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:56.646098 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa225a40-4a5c-4000-9b56-abb09f697934 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:56.646434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa225a40-4a5c-4000-9b56-abb09f697934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:56.680717 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-aa225a40-4a5c-4000-9b56-abb09f697934 None None] MFA Rules not processed for user `75e6fa4efa97411b885f1f7b0add735a`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:56.682185 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-aa225a40-4a5c-4000-9b56-abb09f697934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:56.789553 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1186/2379] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:37:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:56.802830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50687394-0e3e-465d-a89b-bafe3eedc9f0 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:56.804717 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-50687394-0e3e-465d-a89b-bafe3eedc9f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:56.865020 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50687394-0e3e-465d-a89b-bafe3eedc9f0 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:56.868800 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b065d-97cb-492f-ba53-3e2ef77de841 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95452d1b892a463d8e9e3eb16ca7c5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:56.869452 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b065d-97cb-492f-ba53-3e2ef77de841 service glance] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:56.869735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b065d-97cb-492f-ba53-3e2ef77de841 service glance] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:56.870007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b065d-97cb-492f-ba53-3e2ef77de841 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:56.871835 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-644b065d-97cb-492f-ba53-3e2ef77de841 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:56.891369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa225a40-4a5c-4000-9b56-abb09f697934 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:56.893071 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-aa225a40-4a5c-4000-9b56-abb09f697934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:56.948255 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644b065d-97cb-492f-ba53-3e2ef77de841 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:56.949932 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644b065d-97cb-492f-ba53-3e2ef77de841 service glance] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:56.962886 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1194/2380] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:57.002551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa225a40-4a5c-4000-9b56-abb09f697934 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:57.009160 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723537e8-57f7-4f0d-bb4a-dfc507aa2c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:57.010193 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723537e8-57f7-4f0d-bb4a-dfc507aa2c99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:57.010399 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723537e8-57f7-4f0d-bb4a-dfc507aa2c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:57.010798 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723537e8-57f7-4f0d-bb4a-dfc507aa2c99 admin admin] PATH_INFO: `/v3/projects/2e17971f56cb4be4bd29fcf253216d04` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:57.012461 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723537e8-57f7-4f0d-bb4a-dfc507aa2c99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e17971f56cb4be4bd29fcf253216d04)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:57.014193 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723537e8-57f7-4f0d-bb4a-dfc507aa2c99 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:57.082188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:57.151267 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-723537e8-57f7-4f0d-bb4a-dfc507aa2c99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e17971f56cb4be4bd29fcf253216d04', 'request_id': 'req-723537e8-57f7-4f0d-bb4a-dfc507aa2c99'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:57.219067 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-723537e8-57f7-4f0d-bb4a-dfc507aa2c99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e17971f56cb4be4bd29fcf253216d04', 'request_id': 'req-723537e8-57f7-4f0d-bb4a-dfc507aa2c99'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:57.226026 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-723537e8-57f7-4f0d-bb4a-dfc507aa2c99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e17971f56cb4be4bd29fcf253216d04', 'request_id': 'req-723537e8-57f7-4f0d-bb4a-dfc507aa2c99'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:57.237806 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-723537e8-57f7-4f0d-bb4a-dfc507aa2c99 admin admin] The token cache is being invalidate because project 2e17971f56cb4be4bd29fcf253216d04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:37:57.238516 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-723537e8-57f7-4f0d-bb4a-dfc507aa2c99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:57.239954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-723537e8-57f7-4f0d-bb4a-dfc507aa2c99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:37:57.254206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1187/2381] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:37:56 2022] DELETE /identity/v3/projects/2e17971f56cb4be4bd29fcf253216d04 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:37:57.256855 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:57.285029 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979d386-a121-4df0-a944-90c467609750 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:57.285631 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979d386-a121-4df0-a944-90c467609750 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:57.286091 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a979d386-a121-4df0-a944-90c467609750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:57.381470 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-a979d386-a121-4df0-a944-90c467609750 None None] MFA Rules not processed for user `07c9688ca035484ba5341859574c2799`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:37:57.384007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a979d386-a121-4df0-a944-90c467609750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:57.501365 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1195/2382] 10.222.0.24 () {58 vars in 1009 bytes} [Fri May 27 03:37:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:37:57.518996 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723537e8-57f7-4f0d-bb4a-dfc507aa2c99 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:57.521325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-723537e8-57f7-4f0d-bb4a-dfc507aa2c99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:57.628985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723537e8-57f7-4f0d-bb4a-dfc507aa2c99 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:57.635753 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41a2dd-7053-4d2e-a6f1-1cb1da74dfe9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d559cd7127490da84185f20684929d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:57.636343 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41a2dd-7053-4d2e-a6f1-1cb1da74dfe9 service swift] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:57.636628 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41a2dd-7053-4d2e-a6f1-1cb1da74dfe9 service swift] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:57.636904 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41a2dd-7053-4d2e-a6f1-1cb1da74dfe9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:57.646005 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41a2dd-7053-4d2e-a6f1-1cb1da74dfe9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:57.648513 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41a2dd-7053-4d2e-a6f1-1cb1da74dfe9 service swift] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:57.656558 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1188/2383] 10.222.0.24 () {60 vars in 1384 bytes} [Fri May 27 03:37:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:37:57.712448 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979d386-a121-4df0-a944-90c467609750 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:57.713873 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a979d386-a121-4df0-a944-90c467609750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:57.768967 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a979d386-a121-4df0-a944-90c467609750 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:57.773358 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9463bb7-da18-4da0-86b6-7dd041dcdca2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:57.773943 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9463bb7-da18-4da0-86b6-7dd041dcdca2 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:57.774226 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9463bb7-da18-4da0-86b6-7dd041dcdca2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:57.774519 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9463bb7-da18-4da0-86b6-7dd041dcdca2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:57.776524 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b9463bb7-da18-4da0-86b6-7dd041dcdca2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:57.900809 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9463bb7-da18-4da0-86b6-7dd041dcdca2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:57.908279 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9463bb7-da18-4da0-86b6-7dd041dcdca2 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:57.949365 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1196/2384] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:37:57 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:59.010051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:59.624974 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:59.911351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:38:00.247775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:38:00.610022 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:38:01.091742 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:38:01.119905 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:38:01.296313 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cab59-10bf-47ce-b5dc-37042db4b2e6 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:38:01.296873 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cab59-10bf-47ce-b5dc-37042db4b2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:38:01.296873 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cab59-10bf-47ce-b5dc-37042db4b2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:38:01.316142 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-721cab59-10bf-47ce-b5dc-37042db4b2e6 None None] MFA Rules not processed for user `07c9688ca035484ba5341859574c2799`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:38:01.317864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-721cab59-10bf-47ce-b5dc-37042db4b2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:38:01.326364 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1189/2385] 10.222.0.24 () {58 vars in 1009 bytes} [Fri May 27 03:38:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:38:01.343215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9463bb7-da18-4da0-86b6-7dd041dcdca2 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:38:01.364704 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9463bb7-da18-4da0-86b6-7dd041dcdca2 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:38:01.375574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd712f-2101-4395-b5d4-7b70a107abcf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d559cd7127490da84185f20684929d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:38:01.376663 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd712f-2101-4395-b5d4-7b70a107abcf service swift] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:38:01.377120 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd712f-2101-4395-b5d4-7b70a107abcf service swift] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:38:01.377559 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd712f-2101-4395-b5d4-7b70a107abcf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:38:01.390378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd712f-2101-4395-b5d4-7b70a107abcf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:38:01.392427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd712f-2101-4395-b5d4-7b70a107abcf service swift] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:38:01.402156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1197/2386] 10.222.0.24 () {60 vars in 1384 bytes} [Fri May 27 03:38:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:38:01.670615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:38:03.344946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:38:04.387141 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:38:05.193618 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:38:05.599581 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:38:06.678581 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:38:06.861387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:38:06.941469 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:38:11.636149 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=26 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:38:11.685239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1175a-634f-4cc7-aa20-c28483ff8fe2 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:38:11.685537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1175a-634f-4cc7-aa20-c28483ff8fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:38:11.685798 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1175a-634f-4cc7-aa20-c28483ff8fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:38:11.705199 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-56b1175a-634f-4cc7-aa20-c28483ff8fe2 None None] MFA Rules not processed for user `c81a63ee15d843bfb857a7f3148015e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:38:11.706702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-56b1175a-634f-4cc7-aa20-c28483ff8fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:38:11.769592 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1190/2387] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:38:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:38:17.013387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=57 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:38:18.158486 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd712f-2101-4395-b5d4-7b70a107abcf service swift] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:38:18.160092 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-30dd712f-2101-4395-b5d4-7b70a107abcf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:38:18.281238 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd712f-2101-4395-b5d4-7b70a107abcf service swift] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:38:18.286190 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20adde37-abd8-4a27-82b3-b5b1bb78801a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f2d0aceef4451a83ee89fac5130a0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:38:18.286828 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20adde37-abd8-4a27-82b3-b5b1bb78801a service placement] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:38:18.287128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20adde37-abd8-4a27-82b3-b5b1bb78801a service placement] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:38:18.287400 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20adde37-abd8-4a27-82b3-b5b1bb78801a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:38:18.289145 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-20adde37-abd8-4a27-82b3-b5b1bb78801a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:38:18.300598 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20adde37-abd8-4a27-82b3-b5b1bb78801a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:38:18.302646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20adde37-abd8-4a27-82b3-b5b1bb78801a service placement] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:38:18.312594 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1198/2388] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:38:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:38:21.803488 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=25 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:38:25.443934 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995124f-d27c-41d1-885d-bd95112425e6 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:38:25.444651 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995124f-d27c-41d1-885d-bd95112425e6 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:38:25.444651 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995124f-d27c-41d1-885d-bd95112425e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:38:25.465210 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-2995124f-d27c-41d1-885d-bd95112425e6 None None] MFA Rules not processed for user `07c9688ca035484ba5341859574c2799`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:38:25.466469 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-2995124f-d27c-41d1-885d-bd95112425e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:38:25.474903 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1191/2389] 10.222.0.24 () {58 vars in 1009 bytes} [Fri May 27 03:38:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:38:25.489554 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20adde37-abd8-4a27-82b3-b5b1bb78801a service placement] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:38:25.501857 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20adde37-abd8-4a27-82b3-b5b1bb78801a service placement] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:38:25.508763 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef8def-eaff-4be6-bf19-a4b7ed236a05 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d559cd7127490da84185f20684929d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:38:25.509432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef8def-eaff-4be6-bf19-a4b7ed236a05 service swift] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:38:25.509716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef8def-eaff-4be6-bf19-a4b7ed236a05 service swift] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:38:25.509990 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef8def-eaff-4be6-bf19-a4b7ed236a05 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:38:25.520312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef8def-eaff-4be6-bf19-a4b7ed236a05 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:38:25.521962 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef8def-eaff-4be6-bf19-a4b7ed236a05 service swift] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:38:25.539063 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1199/2390] 10.222.0.24 () {60 vars in 1384 bytes} [Fri May 27 03:38:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:38:31.370497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebd9ea-6499-4a03-8308-6c1436bdc24b None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:38:31.370497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebd9ea-6499-4a03-8308-6c1436bdc24b None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:38:31.370497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebd9ea-6499-4a03-8308-6c1436bdc24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:38:31.387282 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-33ebd9ea-6499-4a03-8308-6c1436bdc24b None None] MFA Rules not processed for user `07c9688ca035484ba5341859574c2799`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:38:31.388794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-33ebd9ea-6499-4a03-8308-6c1436bdc24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:38:31.396886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1192/2391] 10.222.0.24 () {58 vars in 1009 bytes} [Fri May 27 03:38:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:38:31.432011 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef8def-eaff-4be6-bf19-a4b7ed236a05 service swift] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:38:31.443994 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef8def-eaff-4be6-bf19-a4b7ed236a05 service swift] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:38:31.458584 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37710ff4-21c6-41a9-a3df-97371fcc25b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d559cd7127490da84185f20684929d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:38:31.458584 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37710ff4-21c6-41a9-a3df-97371fcc25b3 service swift] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:38:31.458584 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37710ff4-21c6-41a9-a3df-97371fcc25b3 service swift] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:38:31.458584 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37710ff4-21c6-41a9-a3df-97371fcc25b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:38:31.468439 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37710ff4-21c6-41a9-a3df-97371fcc25b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:38:31.470273 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37710ff4-21c6-41a9-a3df-97371fcc25b3 service swift] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:38:31.489081 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1200/2392] 10.222.0.24 () {60 vars in 1384 bytes} [Fri May 27 03:38:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:38:41.033214 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc5d63-40e4-4c87-9a73-305bfb65a68c None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:38:41.034590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc5d63-40e4-4c87-9a73-305bfb65a68c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:38:41.035969 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc5d63-40e4-4c87-9a73-305bfb65a68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:38:41.074873 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-21fc5d63-40e4-4c87-9a73-305bfb65a68c None None] MFA Rules not processed for user `07c9688ca035484ba5341859574c2799`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:38:41.079186 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-21fc5d63-40e4-4c87-9a73-305bfb65a68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:38:41.104834 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1193/2393] 10.222.0.24 () {58 vars in 967 bytes} [Fri May 27 03:38:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:38:41.147861 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37710ff4-21c6-41a9-a3df-97371fcc25b3 service swift] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:38:41.164288 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37710ff4-21c6-41a9-a3df-97371fcc25b3 service swift] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:38:41.177384 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f68c3-1ed4-4e7e-8e10-1c31cb6893d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d559cd7127490da84185f20684929d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:38:41.178430 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f68c3-1ed4-4e7e-8e10-1c31cb6893d4 service swift] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:38:41.178982 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f68c3-1ed4-4e7e-8e10-1c31cb6893d4 service swift] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:38:41.179976 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f68c3-1ed4-4e7e-8e10-1c31cb6893d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:38:41.191804 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f68c3-1ed4-4e7e-8e10-1c31cb6893d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:38:41.203007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f68c3-1ed4-4e7e-8e10-1c31cb6893d4 service swift] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:38:41.220315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1201/2394] 10.222.0.24 () {60 vars in 1384 bytes} [Fri May 27 03:38:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:38:44.543493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798db98-fd7a-4018-9f95-0c3623645de0 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:38:44.544036 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798db98-fd7a-4018-9f95-0c3623645de0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:38:44.544197 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798db98-fd7a-4018-9f95-0c3623645de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:38:44.576884 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-1798db98-fd7a-4018-9f95-0c3623645de0 None None] MFA Rules not processed for user `b2671aaae84644acb89c3c3595bb5673`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:38:44.578460 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1798db98-fd7a-4018-9f95-0c3623645de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:38:44.655737 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1194/2395] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:38:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:38:50.465995 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f4802-00fc-4b85-8104-1781714c730b None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:38:50.466898 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f4802-00fc-4b85-8104-1781714c730b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:38:50.466898 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f4802-00fc-4b85-8104-1781714c730b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:38:50.473690 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=60 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:38:50.485985 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-883f4802-00fc-4b85-8104-1781714c730b None None] MFA Rules not processed for user `07c9688ca035484ba5341859574c2799`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:38:50.487255 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-883f4802-00fc-4b85-8104-1781714c730b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:38:50.494773 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1202/2396] 10.222.0.24 () {58 vars in 967 bytes} [Fri May 27 03:38:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:38:50.508679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798db98-fd7a-4018-9f95-0c3623645de0 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:38:50.520557 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798db98-fd7a-4018-9f95-0c3623645de0 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:38:50.525970 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1120e5a-935e-41e6-9034-16872d9dc612 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d559cd7127490da84185f20684929d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:38:50.526594 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1120e5a-935e-41e6-9034-16872d9dc612 service swift] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:38:50.527285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1120e5a-935e-41e6-9034-16872d9dc612 service swift] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:38:50.527285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1120e5a-935e-41e6-9034-16872d9dc612 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:38:50.536253 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1120e5a-935e-41e6-9034-16872d9dc612 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:38:50.537863 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1120e5a-935e-41e6-9034-16872d9dc612 service swift] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:38:50.546163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1195/2397] 10.222.0.24 () {60 vars in 1384 bytes} [Fri May 27 03:38:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:38:53.936757 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6b70c-c79c-41fd-b8b5-c92f790e0f59 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:38:53.937287 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6b70c-c79c-41fd-b8b5-c92f790e0f59 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:38:53.937287 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6b70c-c79c-41fd-b8b5-c92f790e0f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:38:53.956034 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-6cd6b70c-c79c-41fd-b8b5-c92f790e0f59 None None] MFA Rules not processed for user `dac1130950b8457ea3a9349346437357`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:38:53.957831 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6cd6b70c-c79c-41fd-b8b5-c92f790e0f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:38:54.012943 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1203/2398] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:38:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:38:54.745929 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1120e5a-935e-41e6-9034-16872d9dc612 service swift] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:38:54.748002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e1120e5a-935e-41e6-9034-16872d9dc612 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:38:54.754803 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=41 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:38:54.817574 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1120e5a-935e-41e6-9034-16872d9dc612 service swift] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:38:54.822358 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987bccb-1333-4a5e-b957-7f0bff219596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:38:54.823273 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987bccb-1333-4a5e-b957-7f0bff219596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:38:54.823697 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987bccb-1333-4a5e-b957-7f0bff219596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:38:54.823988 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987bccb-1333-4a5e-b957-7f0bff219596 admin admin] PATH_INFO: `/v3/users/1d76f23d7b614fac857a9aefd2d345b6` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:38:54.825315 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987bccb-1333-4a5e-b957-7f0bff219596 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d76f23d7b614fac857a9aefd2d345b6)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:38:54.827454 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987bccb-1333-4a5e-b957-7f0bff219596 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:38:55.065553 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-9987bccb-1333-4a5e-b957-7f0bff219596 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d76f23d7b614fac857a9aefd2d345b6'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:38:55.156517 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-9987bccb-1333-4a5e-b957-7f0bff219596 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d76f23d7b614fac857a9aefd2d345b6'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:38:55.157389 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-9987bccb-1333-4a5e-b957-7f0bff219596 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d76f23d7b614fac857a9aefd2d345b6'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:38:55.167194 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-9987bccb-1333-4a5e-b957-7f0bff219596 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d76f23d7b614fac857a9aefd2d345b6'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:38:55.171494 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-9987bccb-1333-4a5e-b957-7f0bff219596 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d76f23d7b614fac857a9aefd2d345b6'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:38:55.197346 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1196/2399] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:38:54 2022] DELETE /identity/v3/users/1d76f23d7b614fac857a9aefd2d345b6 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:38:55.474146 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6b70c-c79c-41fd-b8b5-c92f790e0f59 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:38:55.486977 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd6b70c-c79c-41fd-b8b5-c92f790e0f59 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:38:55.545074 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c27f1-e3b8-4d25-9bcd-c953f8a33db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:38:55.546029 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c27f1-e3b8-4d25-9bcd-c953f8a33db9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:38:55.546253 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c27f1-e3b8-4d25-9bcd-c953f8a33db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:38:55.546548 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c27f1-e3b8-4d25-9bcd-c953f8a33db9 admin admin] PATH_INFO: `/v3/projects/09b89ed90acc4dd2aa3bca17bb1a1fe9` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:38:55.547785 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c27f1-e3b8-4d25-9bcd-c953f8a33db9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09b89ed90acc4dd2aa3bca17bb1a1fe9)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:38:55.549694 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c27f1-e3b8-4d25-9bcd-c953f8a33db9 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:38:55.662168 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-9d8c27f1-e3b8-4d25-9bcd-c953f8a33db9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09b89ed90acc4dd2aa3bca17bb1a1fe9', 'request_id': 'req-9d8c27f1-e3b8-4d25-9bcd-c953f8a33db9'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:38:55.773289 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-9d8c27f1-e3b8-4d25-9bcd-c953f8a33db9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09b89ed90acc4dd2aa3bca17bb1a1fe9', 'request_id': 'req-9d8c27f1-e3b8-4d25-9bcd-c953f8a33db9'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:38:55.780274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-9d8c27f1-e3b8-4d25-9bcd-c953f8a33db9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09b89ed90acc4dd2aa3bca17bb1a1fe9', 'request_id': 'req-9d8c27f1-e3b8-4d25-9bcd-c953f8a33db9'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:38:55.792943 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-9d8c27f1-e3b8-4d25-9bcd-c953f8a33db9 admin admin] The token cache is being invalidate because project 09b89ed90acc4dd2aa3bca17bb1a1fe9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:38:55.793228 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-9d8c27f1-e3b8-4d25-9bcd-c953f8a33db9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:38:55.793876 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-9d8c27f1-e3b8-4d25-9bcd-c953f8a33db9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:38:55.798965 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1204/2400] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:38:55 2022] DELETE /identity/v3/projects/09b89ed90acc4dd2aa3bca17bb1a1fe9 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:38:55.801538 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74db93a-2329-409a-94bd-0a5d3d5907e1 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:38:55.802052 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74db93a-2329-409a-94bd-0a5d3d5907e1 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:38:55.802147 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74db93a-2329-409a-94bd-0a5d3d5907e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:38:55.820818 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-f74db93a-2329-409a-94bd-0a5d3d5907e1 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:38:55.822015 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f74db93a-2329-409a-94bd-0a5d3d5907e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:38:55.879766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1197/2401] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:38:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:38:55.887674 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db15b6-3726-4477-a5d7-7f69a1843a9e None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:38:55.887961 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db15b6-3726-4477-a5d7-7f69a1843a9e None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:38:55.888295 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db15b6-3726-4477-a5d7-7f69a1843a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:38:55.907192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-20db15b6-3726-4477-a5d7-7f69a1843a9e None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:38:55.908465 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-20db15b6-3726-4477-a5d7-7f69a1843a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:38:55.917436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1205/2402] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:38:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:38:55.921569 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74db93a-2329-409a-94bd-0a5d3d5907e1 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:38:55.931664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74db93a-2329-409a-94bd-0a5d3d5907e1 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:38:55.936444 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb441b88-357e-4c65-ae29-48718d0eef21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:38:55.937098 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb441b88-357e-4c65-ae29-48718d0eef21 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:38:55.937402 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb441b88-357e-4c65-ae29-48718d0eef21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:38:55.937674 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb441b88-357e-4c65-ae29-48718d0eef21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:38:55.938423 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb441b88-357e-4c65-ae29-48718d0eef21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:38:55.938733 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb441b88-357e-4c65-ae29-48718d0eef21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:38:55.940397 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb441b88-357e-4c65-ae29-48718d0eef21 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:38:55.951760 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1198/2403] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:38:55.956413 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db15b6-3726-4477-a5d7-7f69a1843a9e None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:38:55.966066 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db15b6-3726-4477-a5d7-7f69a1843a9e None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:38:55.970668 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247738b-5ba7-4ce4-b02a-52ec66654d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:38:55.971339 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247738b-5ba7-4ce4-b02a-52ec66654d78 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:38:55.971620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247738b-5ba7-4ce4-b02a-52ec66654d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:38:55.972406 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247738b-5ba7-4ce4-b02a-52ec66654d78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:38:55.973107 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247738b-5ba7-4ce4-b02a-52ec66654d78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:38:55.975053 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247738b-5ba7-4ce4-b02a-52ec66654d78 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:38:56.103160 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1206/2404] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:38:55 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:38:56.107168 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb441b88-357e-4c65-ae29-48718d0eef21 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:38:56.117642 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb441b88-357e-4c65-ae29-48718d0eef21 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:38:56.122068 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:38:56.176708 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c737bb7-8c91-4b51-a6c6-1d4acaec2e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:38:56.177506 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c737bb7-8c91-4b51-a6c6-1d4acaec2e7a admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:38:56.177953 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c737bb7-8c91-4b51-a6c6-1d4acaec2e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:38:56.178333 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c737bb7-8c91-4b51-a6c6-1d4acaec2e7a admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:38:56.179196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c737bb7-8c91-4b51-a6c6-1d4acaec2e7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:38:56.181387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c737bb7-8c91-4b51-a6c6-1d4acaec2e7a admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:38:56.288629 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1199/2405] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:38:56 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:38:56.292587 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247738b-5ba7-4ce4-b02a-52ec66654d78 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:38:56.298025 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:38:56.303454 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247738b-5ba7-4ce4-b02a-52ec66654d78 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:38:56.308483 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef69fa3-0a96-4d84-85ad-7e8878051cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:38:56.309149 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef69fa3-0a96-4d84-85ad-7e8878051cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:38:56.309428 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef69fa3-0a96-4d84-85ad-7e8878051cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:38:56.309701 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef69fa3-0a96-4d84-85ad-7e8878051cd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:38:56.310420 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef69fa3-0a96-4d84-85ad-7e8878051cd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:38:56.312065 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef69fa3-0a96-4d84-85ad-7e8878051cd5 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:38:56.322754 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1207/2406] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:38:56.326800 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c737bb7-8c91-4b51-a6c6-1d4acaec2e7a admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:38:56.338151 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c737bb7-8c91-4b51-a6c6-1d4acaec2e7a admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:38:56.342789 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4979b93-b77f-4de1-ad39-e4e7d5d7f2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:38:56.343491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4979b93-b77f-4de1-ad39-e4e7d5d7f2e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:38:56.343776 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4979b93-b77f-4de1-ad39-e4e7d5d7f2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:38:56.344054 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4979b93-b77f-4de1-ad39-e4e7d5d7f2e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:38:56.344785 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4979b93-b77f-4de1-ad39-e4e7d5d7f2e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:38:56.346387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4979b93-b77f-4de1-ad39-e4e7d5d7f2e5 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:38:56.357633 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1200/2407] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:38:56.361713 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef69fa3-0a96-4d84-85ad-7e8878051cd5 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:38:56.372032 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef69fa3-0a96-4d84-85ad-7e8878051cd5 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:38:56.376813 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f470145-b12a-455a-9495-2ea429662127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:38:56.377750 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f470145-b12a-455a-9495-2ea429662127 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:38:56.378027 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f470145-b12a-455a-9495-2ea429662127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:38:56.378334 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f470145-b12a-455a-9495-2ea429662127 admin admin] PATH_INFO: `/v3/projects/bf23f21136c54e2e8fce1a2c0bf8a1a0/users/a3355d76901f46a7a259a5759aff706c/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:38:56.392938 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f470145-b12a-455a-9495-2ea429662127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf23f21136c54e2e8fce1a2c0bf8a1a0, user_id=a3355d76901f46a7a259a5759aff706c, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:38:56.396136 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f470145-b12a-455a-9495-2ea429662127 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:38:56.499192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1208/2408] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:38:56 2022] PUT /identity/v3/projects/bf23f21136c54e2e8fce1a2c0bf8a1a0/users/a3355d76901f46a7a259a5759aff706c/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:38:56.516272 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0e1f6-e447-4022-b439-1309ace26a66 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:38:56.516571 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0e1f6-e447-4022-b439-1309ace26a66 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:38:56.516851 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0e1f6-e447-4022-b439-1309ace26a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:38:56.545235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-60e0e1f6-e447-4022-b439-1309ace26a66 None None] MFA Rules not processed for user `a3355d76901f46a7a259a5759aff706c`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:38:56.546434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-60e0e1f6-e447-4022-b439-1309ace26a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:38:56.616363 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1201/2409] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:38:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:38:56.629289 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f470145-b12a-455a-9495-2ea429662127 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:38:56.634226 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1f470145-b12a-455a-9495-2ea429662127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:38:56.691836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f470145-b12a-455a-9495-2ea429662127 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:38:56.695708 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7546a8-3fa7-4357-aa3f-9737413b894e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:38:56.696360 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7546a8-3fa7-4357-aa3f-9737413b894e service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:38:56.696646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7546a8-3fa7-4357-aa3f-9737413b894e service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:38:56.696922 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7546a8-3fa7-4357-aa3f-9737413b894e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:38:56.706225 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7546a8-3fa7-4357-aa3f-9737413b894e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:38:56.707739 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7546a8-3fa7-4357-aa3f-9737413b894e service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:38:56.716936 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1209/2410] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:38:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:39:00.025643 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0e1f6-e447-4022-b439-1309ace26a66 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:39:00.096095 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e0e1f6-e447-4022-b439-1309ace26a66 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:39:00.101215 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a942e-0396-4117-a59d-232c4bb10437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:39:00.102262 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a942e-0396-4117-a59d-232c4bb10437 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:39:00.102636 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a942e-0396-4117-a59d-232c4bb10437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:39:00.103032 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a942e-0396-4117-a59d-232c4bb10437 admin admin] PATH_INFO: `/v3/users/a3355d76901f46a7a259a5759aff706c` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:39:00.104291 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a942e-0396-4117-a59d-232c4bb10437 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3355d76901f46a7a259a5759aff706c)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:39:00.106346 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a942e-0396-4117-a59d-232c4bb10437 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:39:00.130556 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:39:00.331834 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b56a942e-0396-4117-a59d-232c4bb10437 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3355d76901f46a7a259a5759aff706c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:39:00.397697 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b56a942e-0396-4117-a59d-232c4bb10437 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3355d76901f46a7a259a5759aff706c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:39:00.398791 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b56a942e-0396-4117-a59d-232c4bb10437 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3355d76901f46a7a259a5759aff706c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:39:00.406348 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b56a942e-0396-4117-a59d-232c4bb10437 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3355d76901f46a7a259a5759aff706c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:39:00.407047 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b56a942e-0396-4117-a59d-232c4bb10437 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3355d76901f46a7a259a5759aff706c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:39:00.428506 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1202/2411] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:39:00 2022] DELETE /identity/v3/users/a3355d76901f46a7a259a5759aff706c => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:39:00.433129 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ecf86-39d5-418b-bc11-f8db163caef3 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:39:00.433583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ecf86-39d5-418b-bc11-f8db163caef3 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:39:00.433771 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ecf86-39d5-418b-bc11-f8db163caef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:39:00.452018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-5b7ecf86-39d5-418b-bc11-f8db163caef3 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:39:00.453276 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5b7ecf86-39d5-418b-bc11-f8db163caef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:39:00.513774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1210/2412] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:39:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:39:00.523508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a942e-0396-4117-a59d-232c4bb10437 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:39:00.524925 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b56a942e-0396-4117-a59d-232c4bb10437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:39:00.583713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a942e-0396-4117-a59d-232c4bb10437 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:39:00.587588 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec23cc42-7f07-465c-96c4-b30eca65ec6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:39:00.588246 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23cc42-7f07-465c-96c4-b30eca65ec6f service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:39:00.588535 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23cc42-7f07-465c-96c4-b30eca65ec6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:39:00.588812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23cc42-7f07-465c-96c4-b30eca65ec6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:39:00.598615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec23cc42-7f07-465c-96c4-b30eca65ec6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:39:00.600206 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec23cc42-7f07-465c-96c4-b30eca65ec6f service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:39:00.609761 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1203/2413] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:39:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:39:01.058460 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ecf86-39d5-418b-bc11-f8db163caef3 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:39:01.058797 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5b7ecf86-39d5-418b-bc11-f8db163caef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:39:01.071513 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ecf86-39d5-418b-bc11-f8db163caef3 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:39:01.078036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e261bb9-4942-4338-b717-6211ec7a2c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:39:01.079986 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e261bb9-4942-4338-b717-6211ec7a2c95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:39:01.079986 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e261bb9-4942-4338-b717-6211ec7a2c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:39:01.079986 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e261bb9-4942-4338-b717-6211ec7a2c95 admin admin] PATH_INFO: `/v3/projects/bf23f21136c54e2e8fce1a2c0bf8a1a0` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:39:01.084129 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e261bb9-4942-4338-b717-6211ec7a2c95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf23f21136c54e2e8fce1a2c0bf8a1a0)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:39:01.086516 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e261bb9-4942-4338-b717-6211ec7a2c95 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:39:01.205741 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5e261bb9-4942-4338-b717-6211ec7a2c95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf23f21136c54e2e8fce1a2c0bf8a1a0', 'request_id': 'req-5e261bb9-4942-4338-b717-6211ec7a2c95'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:39:01.312073 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5e261bb9-4942-4338-b717-6211ec7a2c95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf23f21136c54e2e8fce1a2c0bf8a1a0', 'request_id': 'req-5e261bb9-4942-4338-b717-6211ec7a2c95'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:39:01.318652 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5e261bb9-4942-4338-b717-6211ec7a2c95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf23f21136c54e2e8fce1a2c0bf8a1a0', 'request_id': 'req-5e261bb9-4942-4338-b717-6211ec7a2c95'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:39:01.319922 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:39:01.330998 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5e261bb9-4942-4338-b717-6211ec7a2c95 admin admin] The token cache is being invalidate because project bf23f21136c54e2e8fce1a2c0bf8a1a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:39:01.331463 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5e261bb9-4942-4338-b717-6211ec7a2c95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:39:01.332105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5e261bb9-4942-4338-b717-6211ec7a2c95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:39:01.333884 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1211/2414] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:39:01 2022] DELETE /identity/v3/projects/bf23f21136c54e2e8fce1a2c0bf8a1a0 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:39:01.338610 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c762865-e68a-4231-8a57-d9a034a7dcaf None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:39:01.339134 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c762865-e68a-4231-8a57-d9a034a7dcaf None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:39:01.339231 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c762865-e68a-4231-8a57-d9a034a7dcaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:39:01.358204 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-5c762865-e68a-4231-8a57-d9a034a7dcaf None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:39:01.359504 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-5c762865-e68a-4231-8a57-d9a034a7dcaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:39:01.426605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1204/2415] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:39:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:39:01.428292 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2119094-d688-4da4-b96d-e50600a11155 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:39:01.428653 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2119094-d688-4da4-b96d-e50600a11155 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:39:01.441185 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2119094-d688-4da4-b96d-e50600a11155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:39:01.449432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-a2119094-d688-4da4-b96d-e50600a11155 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:39:01.450767 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a2119094-d688-4da4-b96d-e50600a11155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:39:01.459870 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1212/2416] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:39:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:39:01.464055 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c762865-e68a-4231-8a57-d9a034a7dcaf None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:39:01.474018 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c762865-e68a-4231-8a57-d9a034a7dcaf None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:39:01.478702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921e56d-c73c-44ec-90a2-b57b6b3ab4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:39:01.479387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921e56d-c73c-44ec-90a2-b57b6b3ab4a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:39:01.482090 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921e56d-c73c-44ec-90a2-b57b6b3ab4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:39:01.482090 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921e56d-c73c-44ec-90a2-b57b6b3ab4a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:39:01.482090 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e921e56d-c73c-44ec-90a2-b57b6b3ab4a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:39:01.482090 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e921e56d-c73c-44ec-90a2-b57b6b3ab4a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:39:01.482539 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e921e56d-c73c-44ec-90a2-b57b6b3ab4a8 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:39:01.493994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1205/2417] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:39:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:39:01.498147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2119094-d688-4da4-b96d-e50600a11155 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:39:01.508097 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2119094-d688-4da4-b96d-e50600a11155 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:39:01.513334 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4d6de-4e7f-48aa-8f01-bc8a0c2506b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:39:01.513979 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4d6de-4e7f-48aa-8f01-bc8a0c2506b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:39:01.514261 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4d6de-4e7f-48aa-8f01-bc8a0c2506b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:39:01.514562 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4d6de-4e7f-48aa-8f01-bc8a0c2506b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:39:01.515297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc4d6de-4e7f-48aa-8f01-bc8a0c2506b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:39:01.517185 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc4d6de-4e7f-48aa-8f01-bc8a0c2506b7 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:39:01.610006 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1213/2418] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:39:01 2022] POST /identity/v3/projects => generated 369 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:39:01.614593 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921e56d-c73c-44ec-90a2-b57b6b3ab4a8 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:39:01.624825 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921e56d-c73c-44ec-90a2-b57b6b3ab4a8 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:39:01.649864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:39:01.681119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8794b-86aa-4366-a65c-b0871f88b331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:39:01.681782 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8794b-86aa-4366-a65c-b0871f88b331 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:39:01.682065 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8794b-86aa-4366-a65c-b0871f88b331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:39:01.682340 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8794b-86aa-4366-a65c-b0871f88b331 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:39:01.683046 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b8794b-86aa-4366-a65c-b0871f88b331 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:39:01.684982 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b8794b-86aa-4366-a65c-b0871f88b331 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:39:01.809653 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1206/2419] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:39:01 2022] POST /identity/v3/users => generated 285 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:39:01.813682 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4d6de-4e7f-48aa-8f01-bc8a0c2506b7 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:39:01.824496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4d6de-4e7f-48aa-8f01-bc8a0c2506b7 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:39:01.828994 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2e048-ba18-42cb-a014-746e806b660b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:39:01.829643 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2e048-ba18-42cb-a014-746e806b660b admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:39:01.830194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2e048-ba18-42cb-a014-746e806b660b admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:39:01.830450 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2e048-ba18-42cb-a014-746e806b660b admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:39:01.831245 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d2e048-ba18-42cb-a014-746e806b660b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:39:01.832964 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d2e048-ba18-42cb-a014-746e806b660b admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:39:01.847050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1214/2420] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:39:01.848052 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8794b-86aa-4366-a65c-b0871f88b331 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:39:01.858159 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8794b-86aa-4366-a65c-b0871f88b331 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:39:01.864101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18587b6b-ea58-484a-9e94-7bff316032a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:39:01.864844 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18587b6b-ea58-484a-9e94-7bff316032a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:39:01.865320 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18587b6b-ea58-484a-9e94-7bff316032a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:39:01.865579 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18587b6b-ea58-484a-9e94-7bff316032a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:39:01.866414 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18587b6b-ea58-484a-9e94-7bff316032a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:39:01.868170 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18587b6b-ea58-484a-9e94-7bff316032a1 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:39:01.880174 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1207/2421] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:39:01.898328 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2e048-ba18-42cb-a014-746e806b660b admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:39:01.908782 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2e048-ba18-42cb-a014-746e806b660b admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:39:01.913879 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb34717-1dd4-429c-b22a-bca404d68f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:39:01.914807 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb34717-1dd4-429c-b22a-bca404d68f05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:39:01.915171 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb34717-1dd4-429c-b22a-bca404d68f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:39:01.915447 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb34717-1dd4-429c-b22a-bca404d68f05 admin admin] PATH_INFO: `/v3/projects/5fbbc12c436444d080f3222758eff6a3/users/4ac3dc86eb57487889fb518d18cb277b/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:39:01.930566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb34717-1dd4-429c-b22a-bca404d68f05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fbbc12c436444d080f3222758eff6a3, user_id=4ac3dc86eb57487889fb518d18cb277b, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:39:01.933277 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb34717-1dd4-429c-b22a-bca404d68f05 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:39:02.073257 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1215/2422] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:39:01 2022] PUT /identity/v3/projects/5fbbc12c436444d080f3222758eff6a3/users/4ac3dc86eb57487889fb518d18cb277b/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:39:02.086774 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9384c7-6c4d-4501-84ff-56c9cd910609 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:39:02.087090 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9384c7-6c4d-4501-84ff-56c9cd910609 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:39:02.087372 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9384c7-6c4d-4501-84ff-56c9cd910609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:39:02.114097 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-eb9384c7-6c4d-4501-84ff-56c9cd910609 None None] MFA Rules not processed for user `4ac3dc86eb57487889fb518d18cb277b`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:39:02.115318 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-eb9384c7-6c4d-4501-84ff-56c9cd910609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:39:02.179413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1208/2423] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:39:02 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:39:02.190772 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb34717-1dd4-429c-b22a-bca404d68f05 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:39:02.192539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-0fb34717-1dd4-429c-b22a-bca404d68f05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:39:02.251835 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb34717-1dd4-429c-b22a-bca404d68f05 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:39:02.255774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b44909-b9c1-4b3d-9191-5128130e1010 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95452d1b892a463d8e9e3eb16ca7c5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:39:02.256421 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b44909-b9c1-4b3d-9191-5128130e1010 service glance] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:39:02.256701 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b44909-b9c1-4b3d-9191-5128130e1010 service glance] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:39:02.256978 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b44909-b9c1-4b3d-9191-5128130e1010 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:39:02.265565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b44909-b9c1-4b3d-9191-5128130e1010 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:39:02.267032 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b44909-b9c1-4b3d-9191-5128130e1010 service glance] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:39:02.276477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1216/2424] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:39:02 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:39:06.754559 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9384c7-6c4d-4501-84ff-56c9cd910609 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:39:06.755682 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-eb9384c7-6c4d-4501-84ff-56c9cd910609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:39:06.817853 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9384c7-6c4d-4501-84ff-56c9cd910609 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:39:06.821895 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb9e25-775b-47d0-900b-2f736156dd89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:39:06.822713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb9e25-775b-47d0-900b-2f736156dd89 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:39:06.823129 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb9e25-775b-47d0-900b-2f736156dd89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:39:06.823364 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb9e25-775b-47d0-900b-2f736156dd89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:39:06.825292 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-86cb9e25-775b-47d0-900b-2f736156dd89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:39:06.888492 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:39:06.907847 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb9e25-775b-47d0-900b-2f736156dd89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:39:06.909129 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb9e25-775b-47d0-900b-2f736156dd89 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:39:06.920778 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1209/2425] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:39:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:39:12.272389 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=56,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:39:17.441517 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=7 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:39:33.930670 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b44909-b9c1-4b3d-9191-5128130e1010 service glance] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:39:33.994573 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b44909-b9c1-4b3d-9191-5128130e1010 service glance] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:39:33.999002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e623e-e06f-472e-8e0d-3d714466a872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:39:33.999866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e623e-e06f-472e-8e0d-3d714466a872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:39:34.000150 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e623e-e06f-472e-8e0d-3d714466a872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:39:34.000418 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e623e-e06f-472e-8e0d-3d714466a872 admin admin] PATH_INFO: `/v3/users/4ac3dc86eb57487889fb518d18cb277b` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:39:34.001589 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e623e-e06f-472e-8e0d-3d714466a872 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ac3dc86eb57487889fb518d18cb277b)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:39:34.003536 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e623e-e06f-472e-8e0d-3d714466a872 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:39:34.251405 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a44e623e-e06f-472e-8e0d-3d714466a872 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ac3dc86eb57487889fb518d18cb277b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:39:34.322976 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a44e623e-e06f-472e-8e0d-3d714466a872 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ac3dc86eb57487889fb518d18cb277b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:39:34.335738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a44e623e-e06f-472e-8e0d-3d714466a872 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ac3dc86eb57487889fb518d18cb277b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:39:34.336386 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a44e623e-e06f-472e-8e0d-3d714466a872 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ac3dc86eb57487889fb518d18cb277b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:39:34.343222 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a44e623e-e06f-472e-8e0d-3d714466a872 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ac3dc86eb57487889fb518d18cb277b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:39:34.349575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1217/2426] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:39:33 2022] DELETE /identity/v3/users/4ac3dc86eb57487889fb518d18cb277b => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:39:34.354095 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4c29b-8378-4ed6-b730-450cde73e3c5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:39:34.354591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4c29b-8378-4ed6-b730-450cde73e3c5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:39:34.354720 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4c29b-8378-4ed6-b730-450cde73e3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:39:34.372730 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-ebd4c29b-8378-4ed6-b730-450cde73e3c5 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:39:34.373926 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ebd4c29b-8378-4ed6-b730-450cde73e3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:39:34.431641 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1210/2427] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:39:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:39:34.440836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e623e-e06f-472e-8e0d-3d714466a872 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:39:34.442229 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a44e623e-e06f-472e-8e0d-3d714466a872 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:39:34.500338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e623e-e06f-472e-8e0d-3d714466a872 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:39:34.504030 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f43337-77c1-4cb8-b62f-f5267e6d456e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:39:34.504609 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f43337-77c1-4cb8-b62f-f5267e6d456e service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:39:34.504912 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f43337-77c1-4cb8-b62f-f5267e6d456e service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:39:34.505187 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f43337-77c1-4cb8-b62f-f5267e6d456e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:39:34.514041 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f43337-77c1-4cb8-b62f-f5267e6d456e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:39:34.515528 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f43337-77c1-4cb8-b62f-f5267e6d456e service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:39:34.525439 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1218/2428] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:39:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:39:34.966124 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd4c29b-8378-4ed6-b730-450cde73e3c5 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:39:34.967665 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ebd4c29b-8378-4ed6-b730-450cde73e3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:39:34.979954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd4c29b-8378-4ed6-b730-450cde73e3c5 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:39:34.984689 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893621e-42bf-4511-b6e6-e09a8d2c892e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:39:34.985661 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893621e-42bf-4511-b6e6-e09a8d2c892e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:39:34.986012 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893621e-42bf-4511-b6e6-e09a8d2c892e admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:39:34.986363 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893621e-42bf-4511-b6e6-e09a8d2c892e admin admin] PATH_INFO: `/v3/projects/5fbbc12c436444d080f3222758eff6a3` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:39:34.987833 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b893621e-42bf-4511-b6e6-e09a8d2c892e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fbbc12c436444d080f3222758eff6a3)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:39:34.989608 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b893621e-42bf-4511-b6e6-e09a8d2c892e admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:39:35.152296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b893621e-42bf-4511-b6e6-e09a8d2c892e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fbbc12c436444d080f3222758eff6a3', 'request_id': 'req-b893621e-42bf-4511-b6e6-e09a8d2c892e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:39:35.219146 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b893621e-42bf-4511-b6e6-e09a8d2c892e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fbbc12c436444d080f3222758eff6a3', 'request_id': 'req-b893621e-42bf-4511-b6e6-e09a8d2c892e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:39:35.226665 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b893621e-42bf-4511-b6e6-e09a8d2c892e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fbbc12c436444d080f3222758eff6a3', 'request_id': 'req-b893621e-42bf-4511-b6e6-e09a8d2c892e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:39:35.237122 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b893621e-42bf-4511-b6e6-e09a8d2c892e admin admin] The token cache is being invalidate because project 5fbbc12c436444d080f3222758eff6a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:39:35.237428 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b893621e-42bf-4511-b6e6-e09a8d2c892e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:39:35.238032 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b893621e-42bf-4511-b6e6-e09a8d2c892e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:39:35.240246 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1211/2429] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:39:34 2022] DELETE /identity/v3/projects/5fbbc12c436444d080f3222758eff6a3 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:39:43.397387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5cf6d-53ae-4653-abf2-4a2184cbf632 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:39:43.397926 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5cf6d-53ae-4653-abf2-4a2184cbf632 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:39:43.397926 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5cf6d-53ae-4653-abf2-4a2184cbf632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:39:43.415438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-13f5cf6d-53ae-4653-abf2-4a2184cbf632 None None] MFA Rules not processed for user `104a135932a144269f1f2aa7359098ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:39:43.416652 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-13f5cf6d-53ae-4653-abf2-4a2184cbf632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:39:43.433951 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:39:43.471187 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1219/2430] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:39:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:39:45.234041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:39:53.614504 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=15 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:40:07.710547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893621e-42bf-4511-b6e6-e09a8d2c892e admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:40:07.711991 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b893621e-42bf-4511-b6e6-e09a8d2c892e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:07.767629 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b893621e-42bf-4511-b6e6-e09a8d2c892e admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:40:07.771292 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc0b3d-a33b-4921-b6ac-948469c1e5b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:40:07.771969 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc0b3d-a33b-4921-b6ac-948469c1e5b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:07.772242 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc0b3d-a33b-4921-b6ac-948469c1e5b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:07.772517 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc0b3d-a33b-4921-b6ac-948469c1e5b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:07.774528 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3bcc0b3d-a33b-4921-b6ac-948469c1e5b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:07.848957 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc0b3d-a33b-4921-b6ac-948469c1e5b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:40:07.850434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc0b3d-a33b-4921-b6ac-948469c1e5b1 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:40:07.861299 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1212/2431] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:40:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:40:08.410241 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f5cf6d-53ae-4653-abf2-4a2184cbf632 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:40:08.419773 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f5cf6d-53ae-4653-abf2-4a2184cbf632 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:40:08.423394 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac3bc7-47d4-41d2-9805-2e2e2a2d309e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:40:08.424024 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac3bc7-47d4-41d2-9805-2e2e2a2d309e service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:08.424307 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac3bc7-47d4-41d2-9805-2e2e2a2d309e service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:08.424582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac3bc7-47d4-41d2-9805-2e2e2a2d309e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:08.426364 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b1ac3bc7-47d4-41d2-9805-2e2e2a2d309e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:08.440397 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac3bc7-47d4-41d2-9805-2e2e2a2d309e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:40:08.441776 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ac3bc7-47d4-41d2-9805-2e2e2a2d309e service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:40:08.453636 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1220/2432] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:40:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:40:17.362722 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed83f4-5453-4bd5-b309-706390443076 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:17.363347 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed83f4-5453-4bd5-b309-706390443076 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:17.363347 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed83f4-5453-4bd5-b309-706390443076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:17.384214 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-b9ed83f4-5453-4bd5-b309-706390443076 None None] MFA Rules not processed for user `b2671aaae84644acb89c3c3595bb5673`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:40:17.385265 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b9ed83f4-5453-4bd5-b309-706390443076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:17.444135 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1213/2433] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:40:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:40:18.448680 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=6 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:40:19.574930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac3bc7-47d4-41d2-9805-2e2e2a2d309e service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:40:19.583953 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b1ac3bc7-47d4-41d2-9805-2e2e2a2d309e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:19.587932 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ac3bc7-47d4-41d2-9805-2e2e2a2d309e service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:40:19.591977 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31515238-d318-498f-a128-86e7ceab1963 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:40:19.592615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31515238-d318-498f-a128-86e7ceab1963 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:19.592896 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31515238-d318-498f-a128-86e7ceab1963 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:19.593171 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31515238-d318-498f-a128-86e7ceab1963 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:19.595087 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-31515238-d318-498f-a128-86e7ceab1963 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:19.655250 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31515238-d318-498f-a128-86e7ceab1963 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:40:19.656695 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31515238-d318-498f-a128-86e7ceab1963 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:40:19.665676 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1221/2434] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:40:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:40:21.901504 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102ea42-0e49-46b3-806d-c43fe3155039 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:21.902022 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102ea42-0e49-46b3-806d-c43fe3155039 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:21.902121 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102ea42-0e49-46b3-806d-c43fe3155039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:21.920240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-c102ea42-0e49-46b3-806d-c43fe3155039 None None] MFA Rules not processed for user `dac1130950b8457ea3a9349346437357`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:40:21.921428 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c102ea42-0e49-46b3-806d-c43fe3155039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:21.928397 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1214/2435] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:40:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:40:22.302337 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31515238-d318-498f-a128-86e7ceab1963 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:40:22.310994 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31515238-d318-498f-a128-86e7ceab1963 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:40:22.314537 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f74e1-f296-489f-9717-eedbf7b785e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:40:22.315220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f74e1-f296-489f-9717-eedbf7b785e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:22.315506 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f74e1-f296-489f-9717-eedbf7b785e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:22.315815 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f74e1-f296-489f-9717-eedbf7b785e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:22.318167 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a76f74e1-f296-489f-9717-eedbf7b785e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:22.327765 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f74e1-f296-489f-9717-eedbf7b785e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:40:22.329247 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f74e1-f296-489f-9717-eedbf7b785e3 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:40:22.338103 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1222/2436] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:40:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:40:22.751336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102ea42-0e49-46b3-806d-c43fe3155039 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:40:22.754363 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c102ea42-0e49-46b3-806d-c43fe3155039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:22.801392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:40:22.828156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102ea42-0e49-46b3-806d-c43fe3155039 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:40:22.834050 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e236a8b0-0290-47ed-b499-e96e1b1faa47 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f2d0aceef4451a83ee89fac5130a0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:40:22.834691 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236a8b0-0290-47ed-b499-e96e1b1faa47 service placement] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:22.834999 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236a8b0-0290-47ed-b499-e96e1b1faa47 service placement] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:22.835308 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236a8b0-0290-47ed-b499-e96e1b1faa47 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:22.837108 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e236a8b0-0290-47ed-b499-e96e1b1faa47 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:22.848740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e236a8b0-0290-47ed-b499-e96e1b1faa47 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:40:22.850669 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e236a8b0-0290-47ed-b499-e96e1b1faa47 service placement] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:40:22.861292 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1215/2437] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:40:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:40:26.289780 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f74e1-f296-489f-9717-eedbf7b785e3 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:40:26.304198 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f74e1-f296-489f-9717-eedbf7b785e3 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:40:26.308359 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d54ad-6300-4fd0-a7b8-3cedb094006d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:40:26.309214 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d54ad-6300-4fd0-a7b8-3cedb094006d service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:26.309690 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d54ad-6300-4fd0-a7b8-3cedb094006d service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:26.310110 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d54ad-6300-4fd0-a7b8-3cedb094006d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:26.312296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-fd0d54ad-6300-4fd0-a7b8-3cedb094006d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:26.324707 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d54ad-6300-4fd0-a7b8-3cedb094006d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:40:26.326289 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d54ad-6300-4fd0-a7b8-3cedb094006d service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:40:26.335602 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1223/2438] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:40:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:40:32.856499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=14 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:40:36.331729 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=40 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:40:37.814071 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e236a8b0-0290-47ed-b499-e96e1b1faa47 service placement] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:40:37.815613 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e236a8b0-0290-47ed-b499-e96e1b1faa47 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:37.831343 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e236a8b0-0290-47ed-b499-e96e1b1faa47 service placement] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:40:37.838028 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ee849-caaf-4026-9962-0626cd27730b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:40:37.838680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ee849-caaf-4026-9962-0626cd27730b service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:37.839003 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ee849-caaf-4026-9962-0626cd27730b service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:37.839288 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ee849-caaf-4026-9962-0626cd27730b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:37.841076 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1d3ee849-caaf-4026-9962-0626cd27730b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:37.853816 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ee849-caaf-4026-9962-0626cd27730b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:40:37.855633 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ee849-caaf-4026-9962-0626cd27730b service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:40:37.868351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1216/2439] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:40:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:40:43.621284 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d54ad-6300-4fd0-a7b8-3cedb094006d service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:40:43.634981 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d54ad-6300-4fd0-a7b8-3cedb094006d service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:40:43.640479 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2f2e3-8683-41eb-8b23-d126885c4b92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:40:43.641135 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2f2e3-8683-41eb-8b23-d126885c4b92 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:43.641411 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2f2e3-8683-41eb-8b23-d126885c4b92 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:43.641680 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2f2e3-8683-41eb-8b23-d126885c4b92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:43.643437 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6ee2f2e3-8683-41eb-8b23-d126885c4b92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:43.653728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2f2e3-8683-41eb-8b23-d126885c4b92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:40:43.655391 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2f2e3-8683-41eb-8b23-d126885c4b92 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:40:43.664416 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1224/2440] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:40:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:40:47.863547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=6 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:40:51.541643 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ee849-caaf-4026-9962-0626cd27730b service nova] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:40:51.543956 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1d3ee849-caaf-4026-9962-0626cd27730b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:51.558963 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ee849-caaf-4026-9962-0626cd27730b service nova] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:40:51.564679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a59a60-97b4-42e2-83c3-06cd61b9e364 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:40:51.565334 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a59a60-97b4-42e2-83c3-06cd61b9e364 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:51.565693 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a59a60-97b4-42e2-83c3-06cd61b9e364 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:51.565977 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a59a60-97b4-42e2-83c3-06cd61b9e364 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:51.567822 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-69a59a60-97b4-42e2-83c3-06cd61b9e364 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:51.581490 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a59a60-97b4-42e2-83c3-06cd61b9e364 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:40:51.584428 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a59a60-97b4-42e2-83c3-06cd61b9e364 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:40:51.598004 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1217/2441] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:40:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:40:53.660208 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=6 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:40:53.985802 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2f2e3-8683-41eb-8b23-d126885c4b92 service nova] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:40:54.000452 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2f2e3-8683-41eb-8b23-d126885c4b92 service nova] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:40:54.006702 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008813cb-42e8-4806-8161-be49aab443ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:40:54.007390 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008813cb-42e8-4806-8161-be49aab443ed service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:40:54.007675 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008813cb-42e8-4806-8161-be49aab443ed service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:40:54.007956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008813cb-42e8-4806-8161-be49aab443ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:40:54.009968 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-008813cb-42e8-4806-8161-be49aab443ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:40:54.024371 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008813cb-42e8-4806-8161-be49aab443ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:40:54.025614 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008813cb-42e8-4806-8161-be49aab443ed service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:40:54.040297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1225/2442] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:40:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:41:01.592424 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=6 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:41:03.030202 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f0fdc-9559-42d1-a2e2-f5cac83b53f0 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:03.030769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f0fdc-9559-42d1-a2e2-f5cac83b53f0 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:03.030769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f0fdc-9559-42d1-a2e2-f5cac83b53f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:03.049112 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-cb2f0fdc-9559-42d1-a2e2-f5cac83b53f0 None None] MFA Rules not processed for user `07c9688ca035484ba5341859574c2799`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:03.050303 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-cb2f0fdc-9559-42d1-a2e2-f5cac83b53f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:03.106728 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1218/2443] 10.222.0.24 () {58 vars in 967 bytes} [Fri May 27 03:41:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:41:03.300397 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008813cb-42e8-4806-8161-be49aab443ed service nova] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:03.302084 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-008813cb-42e8-4806-8161-be49aab443ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:03.377419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008813cb-42e8-4806-8161-be49aab443ed service nova] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:03.383801 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd731bd-ad7e-4fe2-bf29-25f800bafb8d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d559cd7127490da84185f20684929d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:03.384638 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd731bd-ad7e-4fe2-bf29-25f800bafb8d service swift] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:03.384921 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd731bd-ad7e-4fe2-bf29-25f800bafb8d service swift] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:03.385195 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd731bd-ad7e-4fe2-bf29-25f800bafb8d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:03.394566 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd731bd-ad7e-4fe2-bf29-25f800bafb8d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:03.396310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd731bd-ad7e-4fe2-bf29-25f800bafb8d service swift] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:03.405635 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1226/2444] 10.222.0.24 () {60 vars in 1384 bytes} [Fri May 27 03:41:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:41:08.654917 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f0fdc-9559-42d1-a2e2-f5cac83b53f0 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:08.667762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f0fdc-9559-42d1-a2e2-f5cac83b53f0 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:08.672962 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2b74b-3d58-42aa-9764-d4bb136ef79b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f2d0aceef4451a83ee89fac5130a0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:08.673678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2b74b-3d58-42aa-9764-d4bb136ef79b service placement] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:08.673966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2b74b-3d58-42aa-9764-d4bb136ef79b service placement] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:08.674240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2b74b-3d58-42aa-9764-d4bb136ef79b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:08.676070 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-4ab2b74b-3d58-42aa-9764-d4bb136ef79b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:08.688620 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab2b74b-3d58-42aa-9764-d4bb136ef79b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:08.690226 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab2b74b-3d58-42aa-9764-d4bb136ef79b service placement] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:08.700824 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1219/2445] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:41:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:41:09.716337 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6d8d0-807c-4ce7-97de-b8cb7d1ee631 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:09.717350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6d8d0-807c-4ce7-97de-b8cb7d1ee631 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:09.717785 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6d8d0-807c-4ce7-97de-b8cb7d1ee631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:09.737629 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-73f6d8d0-807c-4ce7-97de-b8cb7d1ee631 None None] MFA Rules not processed for user `07c9688ca035484ba5341859574c2799`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:09.739099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-73f6d8d0-807c-4ce7-97de-b8cb7d1ee631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:09.747856 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1227/2446] 10.222.0.24 () {58 vars in 1009 bytes} [Fri May 27 03:41:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:41:09.760075 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2b74b-3d58-42aa-9764-d4bb136ef79b service placement] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:09.771915 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2b74b-3d58-42aa-9764-d4bb136ef79b service placement] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:09.777627 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4a374-e541-4ff8-bf1f-3c19395a2ec1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d559cd7127490da84185f20684929d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:09.790252 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4a374-e541-4ff8-bf1f-3c19395a2ec1 service swift] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:09.790252 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4a374-e541-4ff8-bf1f-3c19395a2ec1 service swift] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:09.790252 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4a374-e541-4ff8-bf1f-3c19395a2ec1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:09.790252 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a4a374-e541-4ff8-bf1f-3c19395a2ec1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:09.790252 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a4a374-e541-4ff8-bf1f-3c19395a2ec1 service swift] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:09.798390 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1220/2447] 10.222.0.24 () {60 vars in 1384 bytes} [Fri May 27 03:41:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:41:13.706279 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6d8d0-807c-4ce7-97de-b8cb7d1ee631 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:13.708387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-73f6d8d0-807c-4ce7-97de-b8cb7d1ee631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:13.797942 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6d8d0-807c-4ce7-97de-b8cb7d1ee631 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:13.804583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8dc3d9-4156-4c80-9074-a127704a1482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:13.806133 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8dc3d9-4156-4c80-9074-a127704a1482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:13.806434 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8dc3d9-4156-4c80-9074-a127704a1482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:13.806707 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8dc3d9-4156-4c80-9074-a127704a1482 admin admin] PATH_INFO: `/v3/users/c81a63ee15d843bfb857a7f3148015e8` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:13.807978 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8dc3d9-4156-4c80-9074-a127704a1482 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c81a63ee15d843bfb857a7f3148015e8)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:13.810226 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8dc3d9-4156-4c80-9074-a127704a1482 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:14.311935 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-8c8dc3d9-4156-4c80-9074-a127704a1482 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c81a63ee15d843bfb857a7f3148015e8'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:14.552932 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-8c8dc3d9-4156-4c80-9074-a127704a1482 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c81a63ee15d843bfb857a7f3148015e8'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:14.567381 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-8c8dc3d9-4156-4c80-9074-a127704a1482 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c81a63ee15d843bfb857a7f3148015e8'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:14.568153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-8c8dc3d9-4156-4c80-9074-a127704a1482 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c81a63ee15d843bfb857a7f3148015e8'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:14.569413 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:41:14.575632 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-8c8dc3d9-4156-4c80-9074-a127704a1482 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c81a63ee15d843bfb857a7f3148015e8'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:14.581716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1228/2448] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:41:13 2022] DELETE /identity/v3/users/c81a63ee15d843bfb857a7f3148015e8 => generated 0 bytes in 876 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:41:14.919461 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4a374-e541-4ff8-bf1f-3c19395a2ec1 service swift] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:14.942344 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4a374-e541-4ff8-bf1f-3c19395a2ec1 service swift] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:15.025894 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f5fba-81fe-41fa-878a-435eeb1c0f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:15.026698 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f5fba-81fe-41fa-878a-435eeb1c0f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:15.027152 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f5fba-81fe-41fa-878a-435eeb1c0f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:15.031237 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f5fba-81fe-41fa-878a-435eeb1c0f7c admin admin] PATH_INFO: `/v3/projects/99d82d4c868f404e9bc7e932aadf7670` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:15.032636 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f5fba-81fe-41fa-878a-435eeb1c0f7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=99d82d4c868f404e9bc7e932aadf7670)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:15.034961 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f5fba-81fe-41fa-878a-435eeb1c0f7c admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:15.164919 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a72f5fba-81fe-41fa-878a-435eeb1c0f7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99d82d4c868f404e9bc7e932aadf7670', 'request_id': 'req-a72f5fba-81fe-41fa-878a-435eeb1c0f7c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:15.266033 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a72f5fba-81fe-41fa-878a-435eeb1c0f7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99d82d4c868f404e9bc7e932aadf7670', 'request_id': 'req-a72f5fba-81fe-41fa-878a-435eeb1c0f7c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:15.276536 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a72f5fba-81fe-41fa-878a-435eeb1c0f7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99d82d4c868f404e9bc7e932aadf7670', 'request_id': 'req-a72f5fba-81fe-41fa-878a-435eeb1c0f7c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:15.295883 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a72f5fba-81fe-41fa-878a-435eeb1c0f7c admin admin] The token cache is being invalidate because project 99d82d4c868f404e9bc7e932aadf7670 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:41:15.296322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a72f5fba-81fe-41fa-878a-435eeb1c0f7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:15.297191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a72f5fba-81fe-41fa-878a-435eeb1c0f7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:15.300226 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1221/2449] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:41:14 2022] DELETE /identity/v3/projects/99d82d4c868f404e9bc7e932aadf7670 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:41:15.306045 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8dc3d9-4156-4c80-9074-a127704a1482 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:15.307601 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8c8dc3d9-4156-4c80-9074-a127704a1482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:15.325846 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8dc3d9-4156-4c80-9074-a127704a1482 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:15.436328 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c46e5-aa06-41e2-8e93-e8cdd3df73f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:15.437641 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c46e5-aa06-41e2-8e93-e8cdd3df73f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:15.438101 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c46e5-aa06-41e2-8e93-e8cdd3df73f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:15.438543 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c46e5-aa06-41e2-8e93-e8cdd3df73f6 admin admin] PATH_INFO: `/v3/users/75e6fa4efa97411b885f1f7b0add735a` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:15.439965 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c46e5-aa06-41e2-8e93-e8cdd3df73f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75e6fa4efa97411b885f1f7b0add735a)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:15.443866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c46e5-aa06-41e2-8e93-e8cdd3df73f6 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:15.706509 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-657c46e5-aa06-41e2-8e93-e8cdd3df73f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75e6fa4efa97411b885f1f7b0add735a'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:15.778930 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-657c46e5-aa06-41e2-8e93-e8cdd3df73f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75e6fa4efa97411b885f1f7b0add735a'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:15.818747 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-657c46e5-aa06-41e2-8e93-e8cdd3df73f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75e6fa4efa97411b885f1f7b0add735a'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:15.820208 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-657c46e5-aa06-41e2-8e93-e8cdd3df73f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75e6fa4efa97411b885f1f7b0add735a'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:15.836239 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-657c46e5-aa06-41e2-8e93-e8cdd3df73f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75e6fa4efa97411b885f1f7b0add735a'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:15.859883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1229/2450] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:41:15 2022] DELETE /identity/v3/users/75e6fa4efa97411b885f1f7b0add735a => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:41:16.352726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f5fba-81fe-41fa-878a-435eeb1c0f7c admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:16.354209 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a72f5fba-81fe-41fa-878a-435eeb1c0f7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:16.382690 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f5fba-81fe-41fa-878a-435eeb1c0f7c admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:16.404406 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:41:16.488391 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136cc7a-738c-4166-b76b-96ee6e5327fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:16.489474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136cc7a-738c-4166-b76b-96ee6e5327fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:16.490159 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136cc7a-738c-4166-b76b-96ee6e5327fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:16.490543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136cc7a-738c-4166-b76b-96ee6e5327fd admin admin] PATH_INFO: `/v3/projects/f19d9f99f9b94e21b1ce3676c570f24b` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:16.492348 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136cc7a-738c-4166-b76b-96ee6e5327fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f19d9f99f9b94e21b1ce3676c570f24b)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:16.494252 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136cc7a-738c-4166-b76b-96ee6e5327fd admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:16.657343 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c136cc7a-738c-4166-b76b-96ee6e5327fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f19d9f99f9b94e21b1ce3676c570f24b', 'request_id': 'req-c136cc7a-738c-4166-b76b-96ee6e5327fd'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:16.768183 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c136cc7a-738c-4166-b76b-96ee6e5327fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f19d9f99f9b94e21b1ce3676c570f24b', 'request_id': 'req-c136cc7a-738c-4166-b76b-96ee6e5327fd'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:16.775464 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c136cc7a-738c-4166-b76b-96ee6e5327fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f19d9f99f9b94e21b1ce3676c570f24b', 'request_id': 'req-c136cc7a-738c-4166-b76b-96ee6e5327fd'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:16.794258 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c136cc7a-738c-4166-b76b-96ee6e5327fd admin admin] The token cache is being invalidate because project f19d9f99f9b94e21b1ce3676c570f24b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:41:16.794670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c136cc7a-738c-4166-b76b-96ee6e5327fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:16.795422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c136cc7a-738c-4166-b76b-96ee6e5327fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:16.797045 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1222/2451] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:41:16 2022] DELETE /identity/v3/projects/f19d9f99f9b94e21b1ce3676c570f24b => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:41:25.845732 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=58,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:41:26.790581 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:41:32.039363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d84ec8-bebb-42cd-9bad-a98d0988451f None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:32.039994 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d84ec8-bebb-42cd-9bad-a98d0988451f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:32.039994 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d84ec8-bebb-42cd-9bad-a98d0988451f None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:32.041424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1230/2452] 10.222.0.24 () {54 vars in 921 bytes} [Fri May 27 03:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:41:32.047872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f162c-9e99-4032-a723-6e7af7e40995 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:32.048453 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f162c-9e99-4032-a723-6e7af7e40995 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:32.048453 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32f162c-9e99-4032-a723-6e7af7e40995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:32.071432 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-e32f162c-9e99-4032-a723-6e7af7e40995 None None] MFA Rules not processed for user `22a9592143014f3287cd75c60ca17237`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:32.074707 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e32f162c-9e99-4032-a723-6e7af7e40995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:32.162527 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1223/2453] 10.222.0.24 () {58 vars in 1018 bytes} [Fri May 27 03:41:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:41:32.176233 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d84ec8-bebb-42cd-9bad-a98d0988451f None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:32.179998 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-24d84ec8-bebb-42cd-9bad-a98d0988451f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:32.252889 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d84ec8-bebb-42cd-9bad-a98d0988451f None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:32.257057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c8620-7544-4a25-90b4-a604aac2f060 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95452d1b892a463d8e9e3eb16ca7c5ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:32.257722 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c8620-7544-4a25-90b4-a604aac2f060 service glance] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:32.258126 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c8620-7544-4a25-90b4-a604aac2f060 service glance] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:32.258407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c8620-7544-4a25-90b4-a604aac2f060 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:32.269031 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444c8620-7544-4a25-90b4-a604aac2f060 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:32.270473 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444c8620-7544-4a25-90b4-a604aac2f060 service glance] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:32.281309 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1231/2454] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:41:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:41:32.513241 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0a789-1e4d-48d1-8021-f3ea2cd3373d None None] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:32.513535 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0a789-1e4d-48d1-8021-f3ea2cd3373d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:32.513845 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0a789-1e4d-48d1-8021-f3ea2cd3373d None None] PATH_INFO: `/` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:32.516102 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1224/2455] 10.222.0.24 () {54 vars in 921 bytes} [Fri May 27 03:41:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:41:32.522423 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9e2db-1365-44ce-8383-f4402c77abe2 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:32.522723 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9e2db-1365-44ce-8383-f4402c77abe2 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:32.523034 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea9e2db-1365-44ce-8383-f4402c77abe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:32.547384 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-4ea9e2db-1365-44ce-8383-f4402c77abe2 None None] MFA Rules not processed for user `1af0160cd3da4a0bb391b81f083715d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:32.548759 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4ea9e2db-1365-44ce-8383-f4402c77abe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:32.641093 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1232/2456] 10.222.0.24 () {58 vars in 1018 bytes} [Fri May 27 03:41:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:41:32.657677 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d0a789-1e4d-48d1-8021-f3ea2cd3373d None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:32.660121 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-67d0a789-1e4d-48d1-8021-f3ea2cd3373d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:32.678968 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d0a789-1e4d-48d1-8021-f3ea2cd3373d None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:32.687085 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2da88-ebe2-4a37-a77d-7c8544bfdc3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:32.687754 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2da88-ebe2-4a37-a77d-7c8544bfdc3b service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:32.688037 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2da88-ebe2-4a37-a77d-7c8544bfdc3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:32.688317 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2da88-ebe2-4a37-a77d-7c8544bfdc3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:32.697413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2da88-ebe2-4a37-a77d-7c8544bfdc3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:32.699004 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2da88-ebe2-4a37-a77d-7c8544bfdc3b service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:32.707098 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1225/2457] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:41:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:41:32.798401 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694a3a4-0e20-4e2c-8931-fb063e5b27b5 None None] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:32.798735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694a3a4-0e20-4e2c-8931-fb063e5b27b5 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:32.799054 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694a3a4-0e20-4e2c-8931-fb063e5b27b5 None None] PATH_INFO: `/` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:32.800481 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1233/2458] 10.222.0.24 () {54 vars in 921 bytes} [Fri May 27 03:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:41:32.806660 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f209b06-42de-4a4e-8451-da2280c70dce None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:32.806985 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f209b06-42de-4a4e-8451-da2280c70dce None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:32.807270 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f209b06-42de-4a4e-8451-da2280c70dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:32.826725 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-3f209b06-42de-4a4e-8451-da2280c70dce None None] MFA Rules not processed for user `22a9592143014f3287cd75c60ca17237`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:32.827972 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3f209b06-42de-4a4e-8451-da2280c70dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:32.838806 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1226/2459] 10.222.0.24 () {58 vars in 1018 bytes} [Fri May 27 03:41:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:41:32.849821 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694a3a4-0e20-4e2c-8931-fb063e5b27b5 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:32.851741 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d694a3a4-0e20-4e2c-8931-fb063e5b27b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:32.866398 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694a3a4-0e20-4e2c-8931-fb063e5b27b5 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:32.870179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855ed70-4776-4657-952d-0e1036729344 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:32.870897 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855ed70-4776-4657-952d-0e1036729344 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:32.871311 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855ed70-4776-4657-952d-0e1036729344 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:32.871595 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855ed70-4776-4657-952d-0e1036729344 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:32.883309 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855ed70-4776-4657-952d-0e1036729344 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:32.884794 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855ed70-4776-4657-952d-0e1036729344 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:32.896500 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1234/2460] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:41:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:41:33.946744 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f209b06-42de-4a4e-8451-da2280c70dce None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:33.948489 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3f209b06-42de-4a4e-8451-da2280c70dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:34.021047 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f209b06-42de-4a4e-8451-da2280c70dce None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:34.025594 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344ef95-33e8-4347-ad49-da893828c444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:34.026473 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344ef95-33e8-4347-ad49-da893828c444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:34.026754 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344ef95-33e8-4347-ad49-da893828c444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:34.027062 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1344ef95-33e8-4347-ad49-da893828c444 admin admin] PATH_INFO: `/v3/users/b2671aaae84644acb89c3c3595bb5673` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:34.028301 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344ef95-33e8-4347-ad49-da893828c444 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2671aaae84644acb89c3c3595bb5673)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:34.030250 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1344ef95-33e8-4347-ad49-da893828c444 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:34.301163 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1344ef95-33e8-4347-ad49-da893828c444 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2671aaae84644acb89c3c3595bb5673'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:34.356392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1344ef95-33e8-4347-ad49-da893828c444 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2671aaae84644acb89c3c3595bb5673'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:34.357026 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1344ef95-33e8-4347-ad49-da893828c444 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2671aaae84644acb89c3c3595bb5673'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:34.364602 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1344ef95-33e8-4347-ad49-da893828c444 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2671aaae84644acb89c3c3595bb5673'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:34.365324 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1344ef95-33e8-4347-ad49-da893828c444 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2671aaae84644acb89c3c3595bb5673'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:34.373727 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:41:34.383978 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1227/2461] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:41:33 2022] DELETE /identity/v3/users/b2671aaae84644acb89c3c3595bb5673 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:41:34.393708 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855ed70-4776-4657-952d-0e1036729344 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:34.402575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855ed70-4776-4657-952d-0e1036729344 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:34.456375 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3038d217-2110-4b98-b686-1d5ab5c87a1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:34.457031 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038d217-2110-4b98-b686-1d5ab5c87a1b service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:34.457672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038d217-2110-4b98-b686-1d5ab5c87a1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:34.457953 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038d217-2110-4b98-b686-1d5ab5c87a1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:34.459799 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3038d217-2110-4b98-b686-1d5ab5c87a1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:34.518475 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:41:34.530951 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3038d217-2110-4b98-b686-1d5ab5c87a1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:34.535784 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3038d217-2110-4b98-b686-1d5ab5c87a1b service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:34.542579 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1235/2462] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:41:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:41:34.769530 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344ef95-33e8-4347-ad49-da893828c444 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:34.774283 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1344ef95-33e8-4347-ad49-da893828c444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:34.784573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1344ef95-33e8-4347-ad49-da893828c444 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:34.790229 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ce24d-d7e3-4888-b22a-2aedac849233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:34.791140 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ce24d-d7e3-4888-b22a-2aedac849233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:34.791421 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ce24d-d7e3-4888-b22a-2aedac849233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:34.791691 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ce24d-d7e3-4888-b22a-2aedac849233 admin admin] PATH_INFO: `/v3/projects/03b1335715234895a0c1a555b22ee8d3` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:34.793852 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9ce24d-d7e3-4888-b22a-2aedac849233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03b1335715234895a0c1a555b22ee8d3)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:34.795847 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9ce24d-d7e3-4888-b22a-2aedac849233 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:34.953545 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7b9ce24d-d7e3-4888-b22a-2aedac849233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03b1335715234895a0c1a555b22ee8d3', 'request_id': 'req-7b9ce24d-d7e3-4888-b22a-2aedac849233'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:35.041003 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7b9ce24d-d7e3-4888-b22a-2aedac849233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03b1335715234895a0c1a555b22ee8d3', 'request_id': 'req-7b9ce24d-d7e3-4888-b22a-2aedac849233'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:35.047881 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7b9ce24d-d7e3-4888-b22a-2aedac849233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03b1335715234895a0c1a555b22ee8d3', 'request_id': 'req-7b9ce24d-d7e3-4888-b22a-2aedac849233'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:35.060643 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7b9ce24d-d7e3-4888-b22a-2aedac849233 admin admin] The token cache is being invalidate because project 03b1335715234895a0c1a555b22ee8d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:41:35.060878 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7b9ce24d-d7e3-4888-b22a-2aedac849233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:35.061530 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7b9ce24d-d7e3-4888-b22a-2aedac849233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:41:35.063133 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1228/2463] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:41:34 2022] DELETE /identity/v3/projects/03b1335715234895a0c1a555b22ee8d3 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:41:35.068390 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8a5c9-a2c1-4e4f-9642-3eb0c28cd4c4 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:35.068678 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8a5c9-a2c1-4e4f-9642-3eb0c28cd4c4 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:35.068957 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c8a5c9-a2c1-4e4f-9642-3eb0c28cd4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:35.087202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-31c8a5c9-a2c1-4e4f-9642-3eb0c28cd4c4 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:35.088412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-31c8a5c9-a2c1-4e4f-9642-3eb0c28cd4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:35.148868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1236/2464] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:41:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:41:35.156747 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f2c73-9b94-4fd4-bd65-62dd0a230f39 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:35.157137 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f2c73-9b94-4fd4-bd65-62dd0a230f39 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:35.157421 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f2c73-9b94-4fd4-bd65-62dd0a230f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:35.175841 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-026f2c73-9b94-4fd4-bd65-62dd0a230f39 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:35.177032 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-026f2c73-9b94-4fd4-bd65-62dd0a230f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:35.185081 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1229/2465] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:41:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:41:35.189517 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c8a5c9-a2c1-4e4f-9642-3eb0c28cd4c4 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:35.199362 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c8a5c9-a2c1-4e4f-9642-3eb0c28cd4c4 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:35.203872 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f41eb-1d37-41a7-ab58-a3521bf8eacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:35.204519 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f41eb-1d37-41a7-ab58-a3521bf8eacc admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:35.204801 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f41eb-1d37-41a7-ab58-a3521bf8eacc admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:35.205079 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1f41eb-1d37-41a7-ab58-a3521bf8eacc admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:35.205800 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f41eb-1d37-41a7-ab58-a3521bf8eacc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:41:35.206106 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f41eb-1d37-41a7-ab58-a3521bf8eacc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:35.208036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1f41eb-1d37-41a7-ab58-a3521bf8eacc admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:35.219996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1237/2466] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:41:35.224350 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f2c73-9b94-4fd4-bd65-62dd0a230f39 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:35.234541 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f2c73-9b94-4fd4-bd65-62dd0a230f39 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:35.239830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ec135-6a61-4973-bfba-a5eb8153148b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:35.240498 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ec135-6a61-4973-bfba-a5eb8153148b admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:35.240922 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ec135-6a61-4973-bfba-a5eb8153148b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:35.241215 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ec135-6a61-4973-bfba-a5eb8153148b admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:35.242247 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258ec135-6a61-4973-bfba-a5eb8153148b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:35.243880 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258ec135-6a61-4973-bfba-a5eb8153148b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:35.356487 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1230/2467] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:41:35 2022] POST /identity/v3/projects => generated 377 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:41:35.357376 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f41eb-1d37-41a7-ab58-a3521bf8eacc admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:35.367902 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1f41eb-1d37-41a7-ab58-a3521bf8eacc admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:35.431200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb9b12-478e-4dd4-9a18-fd8a8cea5370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:35.431900 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb9b12-478e-4dd4-9a18-fd8a8cea5370 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:35.432170 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb9b12-478e-4dd4-9a18-fd8a8cea5370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:35.432445 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb9b12-478e-4dd4-9a18-fd8a8cea5370 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:35.433139 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb9b12-478e-4dd4-9a18-fd8a8cea5370 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:35.435178 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb9b12-478e-4dd4-9a18-fd8a8cea5370 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:35.545228 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1238/2468] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:41:35 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:41:35.549515 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ec135-6a61-4973-bfba-a5eb8153148b admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:35.560348 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ec135-6a61-4973-bfba-a5eb8153148b admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:35.565586 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9b0f9-c3b8-483d-87e7-e36eae29d24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:35.566401 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9b0f9-c3b8-483d-87e7-e36eae29d24c admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:35.568347 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9b0f9-c3b8-483d-87e7-e36eae29d24c admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:35.568347 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9b0f9-c3b8-483d-87e7-e36eae29d24c admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:35.568347 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9b0f9-c3b8-483d-87e7-e36eae29d24c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:35.570171 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9b0f9-c3b8-483d-87e7-e36eae29d24c admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:35.580681 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1231/2469] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:41:35.585353 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb9b12-478e-4dd4-9a18-fd8a8cea5370 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:35.595598 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb9b12-478e-4dd4-9a18-fd8a8cea5370 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:35.600929 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6cc4d-7d3b-4b72-954b-d52573deab9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:35.601588 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6cc4d-7d3b-4b72-954b-d52573deab9a admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:35.602404 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6cc4d-7d3b-4b72-954b-d52573deab9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:35.602404 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6cc4d-7d3b-4b72-954b-d52573deab9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:35.603031 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6cc4d-7d3b-4b72-954b-d52573deab9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:35.604652 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6cc4d-7d3b-4b72-954b-d52573deab9a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:35.616369 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1239/2470] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:41:35.620664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9b0f9-c3b8-483d-87e7-e36eae29d24c admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:35.632103 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9b0f9-c3b8-483d-87e7-e36eae29d24c admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:35.638042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407de3f-f878-42fa-b2ee-8fb6b2521f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:35.639057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407de3f-f878-42fa-b2ee-8fb6b2521f16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:35.639426 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407de3f-f878-42fa-b2ee-8fb6b2521f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:35.639773 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407de3f-f878-42fa-b2ee-8fb6b2521f16 admin admin] PATH_INFO: `/v3/projects/37f34daf6ac94c41b23e1bc899ab671f/users/2e4436dc550a44269ffbd4760670f551/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:35.656173 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e407de3f-f878-42fa-b2ee-8fb6b2521f16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37f34daf6ac94c41b23e1bc899ab671f, user_id=2e4436dc550a44269ffbd4760670f551, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:35.659010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e407de3f-f878-42fa-b2ee-8fb6b2521f16 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:35.744858 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1232/2471] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:41:35 2022] PUT /identity/v3/projects/37f34daf6ac94c41b23e1bc899ab671f/users/2e4436dc550a44269ffbd4760670f551/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:41:35.746605 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8864e57-652d-47a9-810c-30868e89c097 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:35.746928 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8864e57-652d-47a9-810c-30868e89c097 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:35.747662 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8864e57-652d-47a9-810c-30868e89c097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:35.770325 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-d8864e57-652d-47a9-810c-30868e89c097 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:35.771582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d8864e57-652d-47a9-810c-30868e89c097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:35.823689 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:41:35.837646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1240/2472] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:41:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:41:35.847463 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407de3f-f878-42fa-b2ee-8fb6b2521f16 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:35.849135 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e407de3f-f878-42fa-b2ee-8fb6b2521f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:35.929304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407de3f-f878-42fa-b2ee-8fb6b2521f16 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:35.933919 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50231fb2-31c6-416e-bad1-71225f0921de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:35.934602 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50231fb2-31c6-416e-bad1-71225f0921de service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:35.935007 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50231fb2-31c6-416e-bad1-71225f0921de service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:35.935184 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50231fb2-31c6-416e-bad1-71225f0921de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:35.951304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50231fb2-31c6-416e-bad1-71225f0921de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:35.951304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50231fb2-31c6-416e-bad1-71225f0921de service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:35.955519 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942aca7-6f4a-453c-83a6-4e0f7fbeb8ed None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:35.955812 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942aca7-6f4a-453c-83a6-4e0f7fbeb8ed None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:35.956102 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f942aca7-6f4a-453c-83a6-4e0f7fbeb8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:35.965422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1233/2473] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:41:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:41:35.975939 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-f942aca7-6f4a-453c-83a6-4e0f7fbeb8ed None None] MFA Rules not processed for user `07c9688ca035484ba5341859574c2799`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:35.977150 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f942aca7-6f4a-453c-83a6-4e0f7fbeb8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:36.039520 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1241/2474] 10.222.0.24 () {58 vars in 1009 bytes} [Fri May 27 03:41:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:41:36.053922 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50231fb2-31c6-416e-bad1-71225f0921de service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:36.055386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-50231fb2-31c6-416e-bad1-71225f0921de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:36.062560 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:41:36.134302 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50231fb2-31c6-416e-bad1-71225f0921de service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:36.140949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0b305-fd03-4ea6-ad7b-2165035b0f57 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d559cd7127490da84185f20684929d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:36.142178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0b305-fd03-4ea6-ad7b-2165035b0f57 service swift] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:36.142178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0b305-fd03-4ea6-ad7b-2165035b0f57 service swift] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:36.142178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0b305-fd03-4ea6-ad7b-2165035b0f57 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:36.154481 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f0b305-fd03-4ea6-ad7b-2165035b0f57 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:36.156375 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f0b305-fd03-4ea6-ad7b-2165035b0f57 service swift] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:36.164899 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1234/2475] 10.222.0.24 () {60 vars in 1384 bytes} [Fri May 27 03:41:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:41:42.999647 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01322a9d-6e34-4d01-850b-100ca073bb9c None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:43.004781 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01322a9d-6e34-4d01-850b-100ca073bb9c None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:43.004781 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01322a9d-6e34-4d01-850b-100ca073bb9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:43.047152 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-01322a9d-6e34-4d01-850b-100ca073bb9c None None] MFA Rules not processed for user `2e4436dc550a44269ffbd4760670f551`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:43.049207 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-01322a9d-6e34-4d01-850b-100ca073bb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:43.150894 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1242/2476] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:41:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:41:43.163492 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0b305-fd03-4ea6-ad7b-2165035b0f57 service swift] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:43.165190 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-47f0b305-fd03-4ea6-ad7b-2165035b0f57 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:43.258353 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0b305-fd03-4ea6-ad7b-2165035b0f57 service swift] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:43.264059 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82998a-2c86-45ed-95f0-b1517e65294e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:43.264719 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82998a-2c86-45ed-95f0-b1517e65294e service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:43.265007 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82998a-2c86-45ed-95f0-b1517e65294e service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:43.265300 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82998a-2c86-45ed-95f0-b1517e65294e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:43.275572 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b82998a-2c86-45ed-95f0-b1517e65294e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:43.277275 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b82998a-2c86-45ed-95f0-b1517e65294e service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:43.289226 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1235/2477] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:41:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:41:46.723375 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69476613-cf70-491b-b0dd-678d3ac39532 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:46.724409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69476613-cf70-491b-b0dd-678d3ac39532 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:46.724851 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69476613-cf70-491b-b0dd-678d3ac39532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:46.748786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-69476613-cf70-491b-b0dd-678d3ac39532 None None] MFA Rules not processed for user `2e4436dc550a44269ffbd4760670f551`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:46.752910 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-69476613-cf70-491b-b0dd-678d3ac39532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:46.760830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1243/2478] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:41:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:41:52.590150 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f123c-3625-42d0-adef-828e3f7370a5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:52.590693 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f123c-3625-42d0-adef-828e3f7370a5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:52.590797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f123c-3625-42d0-adef-828e3f7370a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:52.609720 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-2b4f123c-3625-42d0-adef-828e3f7370a5 None None] MFA Rules not processed for user `07c9688ca035484ba5341859574c2799`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:41:52.611299 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-2b4f123c-3625-42d0-adef-828e3f7370a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:52.621759 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1236/2479] 10.222.0.24 () {58 vars in 967 bytes} [Fri May 27 03:41:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:41:52.639453 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69476613-cf70-491b-b0dd-678d3ac39532 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:52.653897 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69476613-cf70-491b-b0dd-678d3ac39532 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:52.662420 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3822c31-c378-4de8-ad81-8969c5ed3af6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d559cd7127490da84185f20684929d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:52.663163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3822c31-c378-4de8-ad81-8969c5ed3af6 service swift] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:52.663454 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3822c31-c378-4de8-ad81-8969c5ed3af6 service swift] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:52.663732 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3822c31-c378-4de8-ad81-8969c5ed3af6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:52.674022 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3822c31-c378-4de8-ad81-8969c5ed3af6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:52.676551 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3822c31-c378-4de8-ad81-8969c5ed3af6 service swift] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:52.686655 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1244/2480] 10.222.0.24 () {60 vars in 1384 bytes} [Fri May 27 03:41:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:41:59.330242 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f123c-3625-42d0-adef-828e3f7370a5 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:41:59.340321 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f123c-3625-42d0-adef-828e3f7370a5 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:41:59.343855 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1f17e-05b9-475a-b2a5-5c5de5a150e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:41:59.344488 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1f17e-05b9-475a-b2a5-5c5de5a150e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:41:59.344769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1f17e-05b9-475a-b2a5-5c5de5a150e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:41:59.345043 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1f17e-05b9-475a-b2a5-5c5de5a150e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:41:59.346801 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-60b1f17e-05b9-475a-b2a5-5c5de5a150e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:41:59.359012 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b1f17e-05b9-475a-b2a5-5c5de5a150e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:41:59.360445 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b1f17e-05b9-475a-b2a5-5c5de5a150e3 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:41:59.369907 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1237/2481] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:41:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:02.685671 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=68 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:03.226446 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3822c31-c378-4de8-ad81-8969c5ed3af6 service swift] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:03.227998 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e3822c31-c378-4de8-ad81-8969c5ed3af6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:03.245864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3822c31-c378-4de8-ad81-8969c5ed3af6 service swift] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:03.252936 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c55d8f-f7b3-474c-9ee3-74dbf8540cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:03.253865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c55d8f-f7b3-474c-9ee3-74dbf8540cab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:03.254151 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c55d8f-f7b3-474c-9ee3-74dbf8540cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:03.254428 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c55d8f-f7b3-474c-9ee3-74dbf8540cab admin admin] PATH_INFO: `/v3/users/dac1130950b8457ea3a9349346437357` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:03.256195 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c55d8f-f7b3-474c-9ee3-74dbf8540cab admin admin] RBAC: Authorizing `identity:delete_user(user_id=dac1130950b8457ea3a9349346437357)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:03.258133 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c55d8f-f7b3-474c-9ee3-74dbf8540cab admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:03.408593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-10c55d8f-f7b3-474c-9ee3-74dbf8540cab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dac1130950b8457ea3a9349346437357'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:03.487609 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-10c55d8f-f7b3-474c-9ee3-74dbf8540cab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dac1130950b8457ea3a9349346437357'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:03.501524 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-10c55d8f-f7b3-474c-9ee3-74dbf8540cab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dac1130950b8457ea3a9349346437357'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:03.502269 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-10c55d8f-f7b3-474c-9ee3-74dbf8540cab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dac1130950b8457ea3a9349346437357'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:03.511137 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-10c55d8f-f7b3-474c-9ee3-74dbf8540cab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dac1130950b8457ea3a9349346437357'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:03.517613 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1245/2482] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:42:03 2022] DELETE /identity/v3/users/dac1130950b8457ea3a9349346437357 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:03.796047 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1f17e-05b9-475a-b2a5-5c5de5a150e3 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:03.814274 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1f17e-05b9-475a-b2a5-5c5de5a150e3 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:03.875879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533fb76-4acb-4b04-aa21-7ae567b602f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:03.876749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533fb76-4acb-4b04-aa21-7ae567b602f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:03.877032 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533fb76-4acb-4b04-aa21-7ae567b602f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:03.877307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533fb76-4acb-4b04-aa21-7ae567b602f4 admin admin] PATH_INFO: `/v3/projects/8ae1382d8f774b5db702d78e3bed622a` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:03.878525 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7533fb76-4acb-4b04-aa21-7ae567b602f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ae1382d8f774b5db702d78e3bed622a)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:03.881181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7533fb76-4acb-4b04-aa21-7ae567b602f4 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:03.882717 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:04.003675 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7533fb76-4acb-4b04-aa21-7ae567b602f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ae1382d8f774b5db702d78e3bed622a', 'request_id': 'req-7533fb76-4acb-4b04-aa21-7ae567b602f4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:04.110368 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7533fb76-4acb-4b04-aa21-7ae567b602f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ae1382d8f774b5db702d78e3bed622a', 'request_id': 'req-7533fb76-4acb-4b04-aa21-7ae567b602f4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:04.118435 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7533fb76-4acb-4b04-aa21-7ae567b602f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ae1382d8f774b5db702d78e3bed622a', 'request_id': 'req-7533fb76-4acb-4b04-aa21-7ae567b602f4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:04.130947 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7533fb76-4acb-4b04-aa21-7ae567b602f4 admin admin] The token cache is being invalidate because project 8ae1382d8f774b5db702d78e3bed622a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:42:04.131408 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7533fb76-4acb-4b04-aa21-7ae567b602f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:04.132188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7533fb76-4acb-4b04-aa21-7ae567b602f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:04.134136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1238/2483] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:42:03 2022] DELETE /identity/v3/projects/8ae1382d8f774b5db702d78e3bed622a => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:04.139246 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f449f-2a53-446e-83a9-d64024db4e44 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:04.139731 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f449f-2a53-446e-83a9-d64024db4e44 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:04.139814 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f449f-2a53-446e-83a9-d64024db4e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:04.159423 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-739f449f-2a53-446e-83a9-d64024db4e44 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:04.160724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-739f449f-2a53-446e-83a9-d64024db4e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:04.225719 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1246/2484] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:42:04.233462 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60faed-6228-408a-bda8-89f061f44147 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:04.234585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60faed-6228-408a-bda8-89f061f44147 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:04.234991 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60faed-6228-408a-bda8-89f061f44147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:04.256740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-eb60faed-6228-408a-bda8-89f061f44147 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:04.258034 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-eb60faed-6228-408a-bda8-89f061f44147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:04.266433 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1239/2485] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:04.270510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f449f-2a53-446e-83a9-d64024db4e44 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:04.281433 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f449f-2a53-446e-83a9-d64024db4e44 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:04.287815 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8805b93-d3d2-4bf1-900b-5f1a1a683dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:04.288664 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8805b93-d3d2-4bf1-900b-5f1a1a683dd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:04.288945 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8805b93-d3d2-4bf1-900b-5f1a1a683dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:04.289215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8805b93-d3d2-4bf1-900b-5f1a1a683dd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:04.289935 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8805b93-d3d2-4bf1-900b-5f1a1a683dd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:42:04.290400 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8805b93-d3d2-4bf1-900b-5f1a1a683dd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:04.292167 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8805b93-d3d2-4bf1-900b-5f1a1a683dd0 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:04.304628 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1247/2486] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:42:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:42:04.309067 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb60faed-6228-408a-bda8-89f061f44147 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:04.319226 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb60faed-6228-408a-bda8-89f061f44147 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:04.325665 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e2e45-c0bc-4771-ae1b-d565d57c937c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:04.326532 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e2e45-c0bc-4771-ae1b-d565d57c937c admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:04.326818 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e2e45-c0bc-4771-ae1b-d565d57c937c admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:04.327117 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e2e45-c0bc-4771-ae1b-d565d57c937c admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:04.327805 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e2e45-c0bc-4771-ae1b-d565d57c937c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:04.329773 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e2e45-c0bc-4771-ae1b-d565d57c937c admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:04.441429 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1240/2487] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:42:04 2022] POST /identity/v3/projects => generated 355 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:42:04.443064 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8805b93-d3d2-4bf1-900b-5f1a1a683dd0 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:04.454477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8805b93-d3d2-4bf1-900b-5f1a1a683dd0 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:04.530944 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df950cf3-84ad-4348-935d-ac772566e765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:04.531958 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df950cf3-84ad-4348-935d-ac772566e765 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:04.532343 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df950cf3-84ad-4348-935d-ac772566e765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:04.532718 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df950cf3-84ad-4348-935d-ac772566e765 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:04.534179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df950cf3-84ad-4348-935d-ac772566e765 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:04.537986 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df950cf3-84ad-4348-935d-ac772566e765 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:04.547126 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:04.670709 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1248/2488] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:42:04 2022] POST /identity/v3/users => generated 278 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:42:04.681063 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e2e45-c0bc-4771-ae1b-d565d57c937c admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:04.691816 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e2e45-c0bc-4771-ae1b-d565d57c937c admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:04.696626 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae312a1-eda9-40f0-81af-019e5f3f5420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:04.697525 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae312a1-eda9-40f0-81af-019e5f3f5420 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:04.697883 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae312a1-eda9-40f0-81af-019e5f3f5420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:04.698219 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae312a1-eda9-40f0-81af-019e5f3f5420 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:04.699000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae312a1-eda9-40f0-81af-019e5f3f5420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:04.700653 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae312a1-eda9-40f0-81af-019e5f3f5420 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:04.712226 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1241/2489] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:42:04.716515 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df950cf3-84ad-4348-935d-ac772566e765 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:04.727248 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df950cf3-84ad-4348-935d-ac772566e765 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:04.731925 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c413f5e1-3a59-406e-a720-03f35dec0529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:04.732582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413f5e1-3a59-406e-a720-03f35dec0529 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:04.732865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413f5e1-3a59-406e-a720-03f35dec0529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:04.733141 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413f5e1-3a59-406e-a720-03f35dec0529 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:04.733988 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c413f5e1-3a59-406e-a720-03f35dec0529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:04.735646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c413f5e1-3a59-406e-a720-03f35dec0529 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:04.749994 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1249/2490] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:42:04.751253 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae312a1-eda9-40f0-81af-019e5f3f5420 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:04.769110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae312a1-eda9-40f0-81af-019e5f3f5420 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:04.774474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e94a79-ceb6-4f70-bb0f-3b36e2f18e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:04.775408 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e94a79-ceb6-4f70-bb0f-3b36e2f18e6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:04.775692 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e94a79-ceb6-4f70-bb0f-3b36e2f18e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:04.775967 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e94a79-ceb6-4f70-bb0f-3b36e2f18e6c admin admin] PATH_INFO: `/v3/projects/41d37f307b1d4ddab0b7f84aacc5f881/users/dfef8488440948c0be895c357db338e0/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:04.791024 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e94a79-ceb6-4f70-bb0f-3b36e2f18e6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=41d37f307b1d4ddab0b7f84aacc5f881, user_id=dfef8488440948c0be895c357db338e0, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:04.793729 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e94a79-ceb6-4f70-bb0f-3b36e2f18e6c admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:04.882810 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1242/2491] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:42:04 2022] PUT /identity/v3/projects/41d37f307b1d4ddab0b7f84aacc5f881/users/dfef8488440948c0be895c357db338e0/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:04.892994 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e1783-3dc2-4972-b51c-cb9525d7634b None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:04.893289 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e1783-3dc2-4972-b51c-cb9525d7634b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:04.893555 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e1783-3dc2-4972-b51c-cb9525d7634b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:04.922392 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-fc7e1783-3dc2-4972-b51c-cb9525d7634b None None] MFA Rules not processed for user `dfef8488440948c0be895c357db338e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:04.923634 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-fc7e1783-3dc2-4972-b51c-cb9525d7634b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:04.996352 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1250/2492] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:04 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:05.000905 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e94a79-ceb6-4f70-bb0f-3b36e2f18e6c admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:05.067121 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e94a79-ceb6-4f70-bb0f-3b36e2f18e6c admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:05.072619 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed8e64-5bd3-4672-9db5-365475f51a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:05.073239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed8e64-5bd3-4672-9db5-365475f51a44 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:05.073524 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed8e64-5bd3-4672-9db5-365475f51a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:05.073802 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed8e64-5bd3-4672-9db5-365475f51a44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:05.074515 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ed8e64-5bd3-4672-9db5-365475f51a44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:05.076462 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ed8e64-5bd3-4672-9db5-365475f51a44 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:05.249195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1243/2493] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:42:04 2022] POST /identity/v3/projects => generated 355 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:42:05.253179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e1783-3dc2-4972-b51c-cb9525d7634b None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:05.264639 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e1783-3dc2-4972-b51c-cb9525d7634b None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:05.322350 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdaf102-5177-41bf-883c-7493e950fe2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:05.323089 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdaf102-5177-41bf-883c-7493e950fe2c admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:05.323513 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdaf102-5177-41bf-883c-7493e950fe2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:05.323797 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdaf102-5177-41bf-883c-7493e950fe2c admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:05.324530 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdaf102-5177-41bf-883c-7493e950fe2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:05.326469 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdaf102-5177-41bf-883c-7493e950fe2c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:05.504858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1251/2494] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:42:05 2022] POST /identity/v3/users => generated 284 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:42:05.508945 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed8e64-5bd3-4672-9db5-365475f51a44 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:05.519262 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed8e64-5bd3-4672-9db5-365475f51a44 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:05.523803 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469be30-0b27-480b-adbb-86df87def13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:05.524462 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469be30-0b27-480b-adbb-86df87def13d admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:05.524743 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469be30-0b27-480b-adbb-86df87def13d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:05.525018 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469be30-0b27-480b-adbb-86df87def13d admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:05.525808 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469be30-0b27-480b-adbb-86df87def13d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:05.527460 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469be30-0b27-480b-adbb-86df87def13d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:05.532509 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:05.538485 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1244/2495] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:42:05.542381 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdaf102-5177-41bf-883c-7493e950fe2c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:05.552343 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdaf102-5177-41bf-883c-7493e950fe2c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:05.557858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225daf11-3922-43bb-bde1-e013f2acebab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:05.558745 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225daf11-3922-43bb-bde1-e013f2acebab admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:05.559069 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225daf11-3922-43bb-bde1-e013f2acebab admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:05.559348 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225daf11-3922-43bb-bde1-e013f2acebab admin admin] PATH_INFO: `/v3/projects/b14e9fd0671b4ce5b4fd8725600167d8/users/027f5d584e0e4d59adfdff5646ccc421/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:05.574327 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225daf11-3922-43bb-bde1-e013f2acebab admin admin] RBAC: Authorizing `identity:create_grant(project_id=b14e9fd0671b4ce5b4fd8725600167d8, user_id=027f5d584e0e4d59adfdff5646ccc421, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:05.577033 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225daf11-3922-43bb-bde1-e013f2acebab admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:05.721927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1252/2496] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:42:05 2022] PUT /identity/v3/projects/b14e9fd0671b4ce5b4fd8725600167d8/users/027f5d584e0e4d59adfdff5646ccc421/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:05.726536 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469be30-0b27-480b-adbb-86df87def13d admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:05.736517 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469be30-0b27-480b-adbb-86df87def13d admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:05.791722 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fc9891-a45e-44e6-bbaf-4992f9283a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:05.792384 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc9891-a45e-44e6-bbaf-4992f9283a5d admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:05.792667 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc9891-a45e-44e6-bbaf-4992f9283a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:05.792943 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fc9891-a45e-44e6-bbaf-4992f9283a5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:05.793662 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fc9891-a45e-44e6-bbaf-4992f9283a5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:05.795343 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fc9891-a45e-44e6-bbaf-4992f9283a5d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:05.805583 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1245/2497] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:42:05.812001 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225daf11-3922-43bb-bde1-e013f2acebab admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:05.820007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225daf11-3922-43bb-bde1-e013f2acebab admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:05.824556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af717e-2cde-45ba-9c5f-1733e4023c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:05.825432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af717e-2cde-45ba-9c5f-1733e4023c0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:05.825716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af717e-2cde-45ba-9c5f-1733e4023c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:05.825990 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af717e-2cde-45ba-9c5f-1733e4023c0b admin admin] PATH_INFO: `/v3/projects/b14e9fd0671b4ce5b4fd8725600167d8/users/027f5d584e0e4d59adfdff5646ccc421/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:05.827994 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62af717e-2cde-45ba-9c5f-1733e4023c0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b14e9fd0671b4ce5b4fd8725600167d8, user_id=027f5d584e0e4d59adfdff5646ccc421, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:05.830653 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62af717e-2cde-45ba-9c5f-1733e4023c0b admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:05.841228 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1253/2498] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:42:05 2022] PUT /identity/v3/projects/b14e9fd0671b4ce5b4fd8725600167d8/users/027f5d584e0e4d59adfdff5646ccc421/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:05.852806 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b86884-ea5c-4bdc-ac82-ea0c11193349 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:05.853177 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b86884-ea5c-4bdc-ac82-ea0c11193349 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:05.853455 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b86884-ea5c-4bdc-ac82-ea0c11193349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:05.879118 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-f7b86884-ea5c-4bdc-ac82-ea0c11193349 None None] MFA Rules not processed for user `027f5d584e0e4d59adfdff5646ccc421`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:05.880304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f7b86884-ea5c-4bdc-ac82-ea0c11193349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:05.945662 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1246/2499] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:05 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:42:05.956872 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af717e-2cde-45ba-9c5f-1733e4023c0b admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:05.959624 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-62af717e-2cde-45ba-9c5f-1733e4023c0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:05.983134 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:06.022653 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af717e-2cde-45ba-9c5f-1733e4023c0b admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:06.026589 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902baa26-8110-4e46-978a-4942fa421764 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:06.027267 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902baa26-8110-4e46-978a-4942fa421764 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:06.027551 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902baa26-8110-4e46-978a-4942fa421764 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:06.027828 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902baa26-8110-4e46-978a-4942fa421764 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:06.037704 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902baa26-8110-4e46-978a-4942fa421764 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:06.039187 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902baa26-8110-4e46-978a-4942fa421764 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:06.049972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1254/2500] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:42:05 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:06.231671 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b86884-ea5c-4bdc-ac82-ea0c11193349 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:06.241184 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b86884-ea5c-4bdc-ac82-ea0c11193349 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:06.244709 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82868a73-462d-4dbe-9b17-19b2e96331fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:06.245337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82868a73-462d-4dbe-9b17-19b2e96331fd service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:06.245614 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82868a73-462d-4dbe-9b17-19b2e96331fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:06.246006 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82868a73-462d-4dbe-9b17-19b2e96331fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:06.302344 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82868a73-462d-4dbe-9b17-19b2e96331fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:06.304381 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82868a73-462d-4dbe-9b17-19b2e96331fd service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:06.317302 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1247/2501] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:42:06 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:16.045756 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=17 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:16.312753 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=91 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:16.487091 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902baa26-8110-4e46-978a-4942fa421764 service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:16.488651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-902baa26-8110-4e46-978a-4942fa421764 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:16.545025 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902baa26-8110-4e46-978a-4942fa421764 service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:16.548628 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec14eb-7ade-47fc-b67a-9e56625e1fd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:16.549273 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec14eb-7ade-47fc-b67a-9e56625e1fd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:16.549530 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec14eb-7ade-47fc-b67a-9e56625e1fd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:16.549802 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ec14eb-7ade-47fc-b67a-9e56625e1fd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:16.551565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-08ec14eb-7ade-47fc-b67a-9e56625e1fd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:16.610629 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec14eb-7ade-47fc-b67a-9e56625e1fd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:16.612120 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ec14eb-7ade-47fc-b67a-9e56625e1fd6 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:16.621065 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1255/2502] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:42:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:23.375264 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82868a73-462d-4dbe-9b17-19b2e96331fd service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:23.384848 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82868a73-462d-4dbe-9b17-19b2e96331fd service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:23.388524 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc77014c-3384-4df2-a2dc-f9b503907d2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:23.389155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77014c-3384-4df2-a2dc-f9b503907d2e service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:23.389435 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77014c-3384-4df2-a2dc-f9b503907d2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:23.389735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77014c-3384-4df2-a2dc-f9b503907d2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:23.391718 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-cc77014c-3384-4df2-a2dc-f9b503907d2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:23.454292 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc77014c-3384-4df2-a2dc-f9b503907d2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:23.455779 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc77014c-3384-4df2-a2dc-f9b503907d2e service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:23.466450 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1248/2503] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:42:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:25.394505 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec14eb-7ade-47fc-b67a-9e56625e1fd6 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:25.406248 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ec14eb-7ade-47fc-b67a-9e56625e1fd6 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:25.411254 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2afc1c-5a83-4d79-8fe9-486b0467229a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:25.412306 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2afc1c-5a83-4d79-8fe9-486b0467229a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:25.412605 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2afc1c-5a83-4d79-8fe9-486b0467229a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:25.412882 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2afc1c-5a83-4d79-8fe9-486b0467229a admin admin] PATH_INFO: `/v3/users/dfef8488440948c0be895c357db338e0` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:25.414479 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2afc1c-5a83-4d79-8fe9-486b0467229a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfef8488440948c0be895c357db338e0)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:25.416558 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2afc1c-5a83-4d79-8fe9-486b0467229a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:25.569534 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3e2afc1c-5a83-4d79-8fe9-486b0467229a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfef8488440948c0be895c357db338e0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:25.606443 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3e2afc1c-5a83-4d79-8fe9-486b0467229a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfef8488440948c0be895c357db338e0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:25.620015 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3e2afc1c-5a83-4d79-8fe9-486b0467229a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfef8488440948c0be895c357db338e0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:25.622180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3e2afc1c-5a83-4d79-8fe9-486b0467229a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfef8488440948c0be895c357db338e0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:25.627680 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3e2afc1c-5a83-4d79-8fe9-486b0467229a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfef8488440948c0be895c357db338e0'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:25.634501 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1256/2504] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:42:25 2022] DELETE /identity/v3/users/dfef8488440948c0be895c357db338e0 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:25.639242 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229d490-d469-444f-9098-4890ee38de61 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:25.639749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229d490-d469-444f-9098-4890ee38de61 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:25.639849 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229d490-d469-444f-9098-4890ee38de61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:25.658137 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-f229d490-d469-444f-9098-4890ee38de61 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:25.659381 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f229d490-d469-444f-9098-4890ee38de61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:25.719648 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1249/2505] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:25.729686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2afc1c-5a83-4d79-8fe9-486b0467229a admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:25.731152 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3e2afc1c-5a83-4d79-8fe9-486b0467229a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:25.764668 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:25.806620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2afc1c-5a83-4d79-8fe9-486b0467229a admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:25.808558 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f484c87b-c75b-43e1-a198-31f20fd75231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:25.809304 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484c87b-c75b-43e1-a198-31f20fd75231 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:25.809573 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484c87b-c75b-43e1-a198-31f20fd75231 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:25.809850 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f484c87b-c75b-43e1-a198-31f20fd75231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:25.821344 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f484c87b-c75b-43e1-a198-31f20fd75231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:25.822812 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f484c87b-c75b-43e1-a198-31f20fd75231 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:25.835594 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1257/2506] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:42:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:42:26.422864 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f229d490-d469-444f-9098-4890ee38de61 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:26.424489 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f229d490-d469-444f-9098-4890ee38de61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:26.448014 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f229d490-d469-444f-9098-4890ee38de61 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:26.453217 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946d106-a2a1-44f9-8cb2-bda93159e303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:26.454394 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946d106-a2a1-44f9-8cb2-bda93159e303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:26.454737 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946d106-a2a1-44f9-8cb2-bda93159e303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:26.459340 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946d106-a2a1-44f9-8cb2-bda93159e303 admin admin] PATH_INFO: `/v3/projects/41d37f307b1d4ddab0b7f84aacc5f881` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:26.459340 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946d106-a2a1-44f9-8cb2-bda93159e303 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41d37f307b1d4ddab0b7f84aacc5f881)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:26.460161 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946d106-a2a1-44f9-8cb2-bda93159e303 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:26.607787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b946d106-a2a1-44f9-8cb2-bda93159e303 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41d37f307b1d4ddab0b7f84aacc5f881', 'request_id': 'req-b946d106-a2a1-44f9-8cb2-bda93159e303'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:26.686969 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b946d106-a2a1-44f9-8cb2-bda93159e303 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41d37f307b1d4ddab0b7f84aacc5f881', 'request_id': 'req-b946d106-a2a1-44f9-8cb2-bda93159e303'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:26.695068 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b946d106-a2a1-44f9-8cb2-bda93159e303 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41d37f307b1d4ddab0b7f84aacc5f881', 'request_id': 'req-b946d106-a2a1-44f9-8cb2-bda93159e303'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:26.707140 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b946d106-a2a1-44f9-8cb2-bda93159e303 admin admin] The token cache is being invalidate because project 41d37f307b1d4ddab0b7f84aacc5f881 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:42:26.707715 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b946d106-a2a1-44f9-8cb2-bda93159e303 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:26.709365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-b946d106-a2a1-44f9-8cb2-bda93159e303 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:26.714585 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1250/2507] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:42:26 2022] DELETE /identity/v3/projects/41d37f307b1d4ddab0b7f84aacc5f881 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:26.719374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f484c87b-c75b-43e1-a198-31f20fd75231 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:26.735455 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f484c87b-c75b-43e1-a198-31f20fd75231 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:26.817296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78669c20-ad4b-4326-9c95-b4e6e8dbe893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:26.840076 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78669c20-ad4b-4326-9c95-b4e6e8dbe893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:26.840076 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78669c20-ad4b-4326-9c95-b4e6e8dbe893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:26.840076 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78669c20-ad4b-4326-9c95-b4e6e8dbe893 admin admin] PATH_INFO: `/v3/users/027f5d584e0e4d59adfdff5646ccc421` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:26.840076 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78669c20-ad4b-4326-9c95-b4e6e8dbe893 admin admin] RBAC: Authorizing `identity:delete_user(user_id=027f5d584e0e4d59adfdff5646ccc421)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:26.840076 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78669c20-ad4b-4326-9c95-b4e6e8dbe893 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:26.895383 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946d106-a2a1-44f9-8cb2-bda93159e303 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:26.897656 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b946d106-a2a1-44f9-8cb2-bda93159e303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:26.921562 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:26.974174 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946d106-a2a1-44f9-8cb2-bda93159e303 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:26.979392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af28259-fca7-4d4f-82fa-67a6dd4ccf34 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f2d0aceef4451a83ee89fac5130a0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:26.980051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af28259-fca7-4d4f-82fa-67a6dd4ccf34 service placement] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:26.980351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af28259-fca7-4d4f-82fa-67a6dd4ccf34 service placement] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:26.980616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af28259-fca7-4d4f-82fa-67a6dd4ccf34 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:26.982374 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-4af28259-fca7-4d4f-82fa-67a6dd4ccf34 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:26.994147 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af28259-fca7-4d4f-82fa-67a6dd4ccf34 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:26.995783 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af28259-fca7-4d4f-82fa-67a6dd4ccf34 service placement] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:27.006269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1251/2508] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:42:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:42:27.060029 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-78669c20-ad4b-4326-9c95-b4e6e8dbe893 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '027f5d584e0e4d59adfdff5646ccc421'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:27.144352 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-78669c20-ad4b-4326-9c95-b4e6e8dbe893 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '027f5d584e0e4d59adfdff5646ccc421'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:27.158254 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-78669c20-ad4b-4326-9c95-b4e6e8dbe893 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '027f5d584e0e4d59adfdff5646ccc421'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:27.159001 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-78669c20-ad4b-4326-9c95-b4e6e8dbe893 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '027f5d584e0e4d59adfdff5646ccc421'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:27.167267 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-78669c20-ad4b-4326-9c95-b4e6e8dbe893 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '027f5d584e0e4d59adfdff5646ccc421'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:27.174219 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1258/2509] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:42:26 2022] DELETE /identity/v3/users/027f5d584e0e4d59adfdff5646ccc421 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:27.752004 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af28259-fca7-4d4f-82fa-67a6dd4ccf34 service placement] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:27.753479 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-4af28259-fca7-4d4f-82fa-67a6dd4ccf34 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:27.767154 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af28259-fca7-4d4f-82fa-67a6dd4ccf34 service placement] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:27.831714 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528ddeb3-f54e-43ae-bd5e-07b0f3504da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:27.831714 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ddeb3-f54e-43ae-bd5e-07b0f3504da4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:27.831714 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ddeb3-f54e-43ae-bd5e-07b0f3504da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:27.831714 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528ddeb3-f54e-43ae-bd5e-07b0f3504da4 admin admin] PATH_INFO: `/v3/projects/b14e9fd0671b4ce5b4fd8725600167d8` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:27.832301 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ddeb3-f54e-43ae-bd5e-07b0f3504da4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b14e9fd0671b4ce5b4fd8725600167d8)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:27.834135 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528ddeb3-f54e-43ae-bd5e-07b0f3504da4 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:27.956898 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-528ddeb3-f54e-43ae-bd5e-07b0f3504da4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b14e9fd0671b4ce5b4fd8725600167d8', 'request_id': 'req-528ddeb3-f54e-43ae-bd5e-07b0f3504da4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:28.035466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-528ddeb3-f54e-43ae-bd5e-07b0f3504da4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b14e9fd0671b4ce5b4fd8725600167d8', 'request_id': 'req-528ddeb3-f54e-43ae-bd5e-07b0f3504da4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:28.042919 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-528ddeb3-f54e-43ae-bd5e-07b0f3504da4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b14e9fd0671b4ce5b4fd8725600167d8', 'request_id': 'req-528ddeb3-f54e-43ae-bd5e-07b0f3504da4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:28.053456 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-528ddeb3-f54e-43ae-bd5e-07b0f3504da4 admin admin] The token cache is being invalidate because project b14e9fd0671b4ce5b4fd8725600167d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:42:28.053742 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-528ddeb3-f54e-43ae-bd5e-07b0f3504da4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:28.054460 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-528ddeb3-f54e-43ae-bd5e-07b0f3504da4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:28.056037 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1252/2510] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:42:27 2022] DELETE /identity/v3/projects/b14e9fd0671b4ce5b4fd8725600167d8 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:42:28.060972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfff36d-58b1-48f1-830b-b2655030db36 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:28.061433 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfff36d-58b1-48f1-830b-b2655030db36 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:28.061553 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfff36d-58b1-48f1-830b-b2655030db36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:28.082131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-5bfff36d-58b1-48f1-830b-b2655030db36 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:28.084204 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5bfff36d-58b1-48f1-830b-b2655030db36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:28.128859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:28.143934 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1259/2511] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:42:28.152277 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab712bc-5a51-4a8f-97d5-0c96c07c6282 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:28.152571 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab712bc-5a51-4a8f-97d5-0c96c07c6282 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:28.152848 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab712bc-5a51-4a8f-97d5-0c96c07c6282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:28.171670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-6ab712bc-5a51-4a8f-97d5-0c96c07c6282 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:28.172895 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6ab712bc-5a51-4a8f-97d5-0c96c07c6282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:28.181089 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1253/2512] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:28.185315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfff36d-58b1-48f1-830b-b2655030db36 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:28.195457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfff36d-58b1-48f1-830b-b2655030db36 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:28.200124 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e76d4-5896-40c3-aa60-7fcb28fa8656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:28.200778 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e76d4-5896-40c3-aa60-7fcb28fa8656 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:28.201060 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e76d4-5896-40c3-aa60-7fcb28fa8656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:28.201333 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e76d4-5896-40c3-aa60-7fcb28fa8656 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:28.202050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8e76d4-5896-40c3-aa60-7fcb28fa8656 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:42:28.202356 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8e76d4-5896-40c3-aa60-7fcb28fa8656 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:28.203989 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8e76d4-5896-40c3-aa60-7fcb28fa8656 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:28.216636 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1260/2513] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:42:28.220887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab712bc-5a51-4a8f-97d5-0c96c07c6282 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:28.230952 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab712bc-5a51-4a8f-97d5-0c96c07c6282 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:28.235775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ec4d9-1066-4090-bfef-f05ed2473e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:28.236417 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ec4d9-1066-4090-bfef-f05ed2473e9b admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:28.236700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ec4d9-1066-4090-bfef-f05ed2473e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:28.236977 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ec4d9-1066-4090-bfef-f05ed2473e9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:28.237691 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ec4d9-1066-4090-bfef-f05ed2473e9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:28.240338 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ec4d9-1066-4090-bfef-f05ed2473e9b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:28.364475 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1254/2514] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:42:28 2022] POST /identity/v3/projects => generated 369 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:42:28.369558 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e76d4-5896-40c3-aa60-7fcb28fa8656 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:28.383166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e76d4-5896-40c3-aa60-7fcb28fa8656 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:28.467146 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dcde8-c20b-43ca-80cf-0135f2e69002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:28.467836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dcde8-c20b-43ca-80cf-0135f2e69002 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:28.468287 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dcde8-c20b-43ca-80cf-0135f2e69002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:28.468846 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dcde8-c20b-43ca-80cf-0135f2e69002 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:28.469631 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14dcde8-c20b-43ca-80cf-0135f2e69002 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:28.471621 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14dcde8-c20b-43ca-80cf-0135f2e69002 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:28.537155 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1261/2515] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:42:28 2022] POST /identity/v3/users => generated 285 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:42:28.538489 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ec4d9-1066-4090-bfef-f05ed2473e9b admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:28.550005 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ec4d9-1066-4090-bfef-f05ed2473e9b admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:28.555465 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b5454-ce13-41bc-8fde-ef77f0ded74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:28.557289 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b5454-ce13-41bc-8fde-ef77f0ded74d admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:28.557289 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b5454-ce13-41bc-8fde-ef77f0ded74d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:28.557289 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700b5454-ce13-41bc-8fde-ef77f0ded74d admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:28.557567 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b5454-ce13-41bc-8fde-ef77f0ded74d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:28.559131 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700b5454-ce13-41bc-8fde-ef77f0ded74d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:28.561549 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:28.573069 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1255/2516] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:42:28.577311 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dcde8-c20b-43ca-80cf-0135f2e69002 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:28.591388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dcde8-c20b-43ca-80cf-0135f2e69002 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:28.596498 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2035306-6885-45fb-9992-6fff3220c263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:28.601352 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2035306-6885-45fb-9992-6fff3220c263 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:28.601352 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2035306-6885-45fb-9992-6fff3220c263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:28.601352 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2035306-6885-45fb-9992-6fff3220c263 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:28.601352 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2035306-6885-45fb-9992-6fff3220c263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:28.601352 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2035306-6885-45fb-9992-6fff3220c263 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:28.613849 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1262/2517] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:42:28.618168 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b5454-ce13-41bc-8fde-ef77f0ded74d admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:28.628061 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700b5454-ce13-41bc-8fde-ef77f0ded74d admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:28.632764 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18480ae-8405-4f54-90ff-1682ee1c76e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:28.633640 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18480ae-8405-4f54-90ff-1682ee1c76e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:28.633921 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18480ae-8405-4f54-90ff-1682ee1c76e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:28.634195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18480ae-8405-4f54-90ff-1682ee1c76e5 admin admin] PATH_INFO: `/v3/projects/3bc83f8444c54b109223dea02aa29cce/users/92bfe9f7351c464aa9682e2f39a00788/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:28.650369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18480ae-8405-4f54-90ff-1682ee1c76e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bc83f8444c54b109223dea02aa29cce, user_id=92bfe9f7351c464aa9682e2f39a00788, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:28.653171 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18480ae-8405-4f54-90ff-1682ee1c76e5 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:28.732430 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1256/2518] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:42:28 2022] PUT /identity/v3/projects/3bc83f8444c54b109223dea02aa29cce/users/92bfe9f7351c464aa9682e2f39a00788/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:28.744252 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e076a3-b990-49ae-b24c-95f522a1d2b3 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:28.744546 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e076a3-b990-49ae-b24c-95f522a1d2b3 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:28.744944 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e076a3-b990-49ae-b24c-95f522a1d2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:28.774015 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-79e076a3-b990-49ae-b24c-95f522a1d2b3 None None] MFA Rules not processed for user `92bfe9f7351c464aa9682e2f39a00788`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:28.775337 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-79e076a3-b990-49ae-b24c-95f522a1d2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:28.841556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1263/2519] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:28 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:28.846053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18480ae-8405-4f54-90ff-1682ee1c76e5 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:28.909458 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18480ae-8405-4f54-90ff-1682ee1c76e5 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:28.914775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d005b0-73a5-4670-840d-fc8814447086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:28.915554 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d005b0-73a5-4670-840d-fc8814447086 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:28.915834 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d005b0-73a5-4670-840d-fc8814447086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:28.916166 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d005b0-73a5-4670-840d-fc8814447086 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:28.916865 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d005b0-73a5-4670-840d-fc8814447086 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:28.918786 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d005b0-73a5-4670-840d-fc8814447086 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:29.001070 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1257/2520] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:42:28 2022] POST /identity/v3/projects => generated 367 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:42:29.006516 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e076a3-b990-49ae-b24c-95f522a1d2b3 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:29.020189 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e076a3-b990-49ae-b24c-95f522a1d2b3 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:29.074988 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:29.093330 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c62673-a657-49d0-8eea-2d8902ca165a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:29.094136 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c62673-a657-49d0-8eea-2d8902ca165a admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:29.094647 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c62673-a657-49d0-8eea-2d8902ca165a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:29.095052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c62673-a657-49d0-8eea-2d8902ca165a admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:29.095878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c62673-a657-49d0-8eea-2d8902ca165a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:29.101146 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c62673-a657-49d0-8eea-2d8902ca165a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:29.197041 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1264/2521] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:42:29 2022] POST /identity/v3/users => generated 290 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:42:29.200407 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d005b0-73a5-4670-840d-fc8814447086 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:29.211932 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d005b0-73a5-4670-840d-fc8814447086 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:29.217781 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b06ae-e744-4ecb-b8b8-7f05ffed6860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:29.218479 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b06ae-e744-4ecb-b8b8-7f05ffed6860 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:29.218762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b06ae-e744-4ecb-b8b8-7f05ffed6860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:29.219200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b06ae-e744-4ecb-b8b8-7f05ffed6860 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:29.220003 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b06ae-e744-4ecb-b8b8-7f05ffed6860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:29.221918 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b06ae-e744-4ecb-b8b8-7f05ffed6860 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:29.236629 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1258/2522] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:42:29.241318 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c62673-a657-49d0-8eea-2d8902ca165a admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:29.251662 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c62673-a657-49d0-8eea-2d8902ca165a admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:29.257203 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16681204-e7f9-413b-a072-3b4da2cfe4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:29.258095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16681204-e7f9-413b-a072-3b4da2cfe4b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:29.258382 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16681204-e7f9-413b-a072-3b4da2cfe4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:29.258656 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16681204-e7f9-413b-a072-3b4da2cfe4b8 admin admin] PATH_INFO: `/v3/projects/eba5357e4f3a4d32810e063fc530841b/users/bcb765bef7b649c29d3a4d64494773a8/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:29.274688 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16681204-e7f9-413b-a072-3b4da2cfe4b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eba5357e4f3a4d32810e063fc530841b, user_id=bcb765bef7b649c29d3a4d64494773a8, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:29.277398 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16681204-e7f9-413b-a072-3b4da2cfe4b8 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:29.369896 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1265/2523] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:42:29 2022] PUT /identity/v3/projects/eba5357e4f3a4d32810e063fc530841b/users/bcb765bef7b649c29d3a4d64494773a8/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:29.375054 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b06ae-e744-4ecb-b8b8-7f05ffed6860 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:29.386184 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b06ae-e744-4ecb-b8b8-7f05ffed6860 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:29.455834 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c8f05-b0c4-4b73-893e-4d7a6c9fe12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:29.456772 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c8f05-b0c4-4b73-893e-4d7a6c9fe12d admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:29.457156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c8f05-b0c4-4b73-893e-4d7a6c9fe12d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:29.457557 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c8f05-b0c4-4b73-893e-4d7a6c9fe12d admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:29.459207 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c8f05-b0c4-4b73-893e-4d7a6c9fe12d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:29.461015 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c8f05-b0c4-4b73-893e-4d7a6c9fe12d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:29.477255 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1259/2524] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:42:29.481194 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16681204-e7f9-413b-a072-3b4da2cfe4b8 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:29.493223 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16681204-e7f9-413b-a072-3b4da2cfe4b8 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:29.507379 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e2a89-e114-41c6-a815-364cb00ca02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:29.508531 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e2a89-e114-41c6-a815-364cb00ca02b admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:29.508980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e2a89-e114-41c6-a815-364cb00ca02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:29.509422 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e2a89-e114-41c6-a815-364cb00ca02b admin admin] PATH_INFO: `/v3/projects/eba5357e4f3a4d32810e063fc530841b/users/bcb765bef7b649c29d3a4d64494773a8/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:29.511962 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e2a89-e114-41c6-a815-364cb00ca02b admin admin] RBAC: Authorizing `identity:create_grant(project_id=eba5357e4f3a4d32810e063fc530841b, user_id=bcb765bef7b649c29d3a4d64494773a8, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:29.515027 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e2a89-e114-41c6-a815-364cb00ca02b admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:29.537554 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1266/2525] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:42:29 2022] PUT /identity/v3/projects/eba5357e4f3a4d32810e063fc530841b/users/bcb765bef7b649c29d3a4d64494773a8/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:29.539487 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319f6d1-611b-4699-84ef-a4851aad80f9 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:29.540670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319f6d1-611b-4699-84ef-a4851aad80f9 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:29.541477 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8319f6d1-611b-4699-84ef-a4851aad80f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:29.583205 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-8319f6d1-611b-4699-84ef-a4851aad80f9 None None] MFA Rules not processed for user `bcb765bef7b649c29d3a4d64494773a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:29.585590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-8319f6d1-611b-4699-84ef-a4851aad80f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:29.628525 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:29.695319 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1260/2526] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:29 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:29.706861 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e2a89-e114-41c6-a815-364cb00ca02b admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:29.709694 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a25e2a89-e114-41c6-a815-364cb00ca02b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:29.775671 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e2a89-e114-41c6-a815-364cb00ca02b admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:29.788762 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc98d3-2745-4f40-b589-fa8b2c6c5be8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:29.789443 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc98d3-2745-4f40-b589-fa8b2c6c5be8 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:29.789725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc98d3-2745-4f40-b589-fa8b2c6c5be8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:29.790002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc98d3-2745-4f40-b589-fa8b2c6c5be8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:29.829678 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc98d3-2745-4f40-b589-fa8b2c6c5be8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:29.834432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc98d3-2745-4f40-b589-fa8b2c6c5be8 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:29.848605 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1267/2527] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:42:29 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:31.545369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319f6d1-611b-4699-84ef-a4851aad80f9 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:31.547042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-8319f6d1-611b-4699-84ef-a4851aad80f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:31.617275 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8319f6d1-611b-4699-84ef-a4851aad80f9 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:31.621776 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d79b9-198b-4b85-af1c-f4f42fabc3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:31.622654 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d79b9-198b-4b85-af1c-f4f42fabc3ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:31.622962 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d79b9-198b-4b85-af1c-f4f42fabc3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:31.623233 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d79b9-198b-4b85-af1c-f4f42fabc3ce admin admin] PATH_INFO: `/v3/users/9001a4b10d544226b3b1453b06883db4` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:31.624402 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d79b9-198b-4b85-af1c-f4f42fabc3ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=9001a4b10d544226b3b1453b06883db4)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:31.626312 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d79b9-198b-4b85-af1c-f4f42fabc3ce admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:31.835369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1b4d79b9-198b-4b85-af1c-f4f42fabc3ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9001a4b10d544226b3b1453b06883db4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:31.946938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1b4d79b9-198b-4b85-af1c-f4f42fabc3ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9001a4b10d544226b3b1453b06883db4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:31.947763 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1b4d79b9-198b-4b85-af1c-f4f42fabc3ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9001a4b10d544226b3b1453b06883db4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:31.959750 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1b4d79b9-198b-4b85-af1c-f4f42fabc3ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9001a4b10d544226b3b1453b06883db4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:31.960517 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1b4d79b9-198b-4b85-af1c-f4f42fabc3ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9001a4b10d544226b3b1453b06883db4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:31.982013 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1261/2528] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:42:31 2022] DELETE /identity/v3/users/9001a4b10d544226b3b1453b06883db4 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:32.255868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc98d3-2745-4f40-b589-fa8b2c6c5be8 service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:32.268299 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc98d3-2745-4f40-b589-fa8b2c6c5be8 service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:32.345989 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482de523-e9c9-47c9-a6db-4bd7be00b155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:32.346863 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482de523-e9c9-47c9-a6db-4bd7be00b155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:32.347186 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482de523-e9c9-47c9-a6db-4bd7be00b155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:32.347459 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482de523-e9c9-47c9-a6db-4bd7be00b155 admin admin] PATH_INFO: `/v3/projects/19592ab2e5e04d7c952aebf73f4fca6e` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:32.348644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482de523-e9c9-47c9-a6db-4bd7be00b155 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19592ab2e5e04d7c952aebf73f4fca6e)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:32.351129 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482de523-e9c9-47c9-a6db-4bd7be00b155 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:32.496811 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:32.504095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-482de523-e9c9-47c9-a6db-4bd7be00b155 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19592ab2e5e04d7c952aebf73f4fca6e', 'request_id': 'req-482de523-e9c9-47c9-a6db-4bd7be00b155'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:32.624128 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-482de523-e9c9-47c9-a6db-4bd7be00b155 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19592ab2e5e04d7c952aebf73f4fca6e', 'request_id': 'req-482de523-e9c9-47c9-a6db-4bd7be00b155'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:32.630597 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-482de523-e9c9-47c9-a6db-4bd7be00b155 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19592ab2e5e04d7c952aebf73f4fca6e', 'request_id': 'req-482de523-e9c9-47c9-a6db-4bd7be00b155'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:32.641531 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-482de523-e9c9-47c9-a6db-4bd7be00b155 admin admin] The token cache is being invalidate because project 19592ab2e5e04d7c952aebf73f4fca6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:42:32.641812 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-482de523-e9c9-47c9-a6db-4bd7be00b155 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:32.642423 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-482de523-e9c9-47c9-a6db-4bd7be00b155 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:32.643981 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1268/2529] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:42:32 2022] DELETE /identity/v3/projects/19592ab2e5e04d7c952aebf73f4fca6e => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:32.648662 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d79b9-198b-4b85-af1c-f4f42fabc3ce admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:32.650148 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1b4d79b9-198b-4b85-af1c-f4f42fabc3ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:32.663214 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d79b9-198b-4b85-af1c-f4f42fabc3ce admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:32.707454 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:32.726478 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc881bb3-2585-4bfb-8f2f-1ad2c2cfbd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:32.728295 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc881bb3-2585-4bfb-8f2f-1ad2c2cfbd68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:32.731910 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc881bb3-2585-4bfb-8f2f-1ad2c2cfbd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:32.731910 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc881bb3-2585-4bfb-8f2f-1ad2c2cfbd68 admin admin] PATH_INFO: `/v3/users/3dae32fa077840d9a053b6bef162615e` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:32.731910 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc881bb3-2585-4bfb-8f2f-1ad2c2cfbd68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dae32fa077840d9a053b6bef162615e)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:32.733727 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc881bb3-2585-4bfb-8f2f-1ad2c2cfbd68 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:33.086400 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-dc881bb3-2585-4bfb-8f2f-1ad2c2cfbd68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dae32fa077840d9a053b6bef162615e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:33.218041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-dc881bb3-2585-4bfb-8f2f-1ad2c2cfbd68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dae32fa077840d9a053b6bef162615e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:33.218744 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-dc881bb3-2585-4bfb-8f2f-1ad2c2cfbd68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dae32fa077840d9a053b6bef162615e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:33.227359 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-dc881bb3-2585-4bfb-8f2f-1ad2c2cfbd68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dae32fa077840d9a053b6bef162615e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:33.228009 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-dc881bb3-2585-4bfb-8f2f-1ad2c2cfbd68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dae32fa077840d9a053b6bef162615e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:33.246418 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1262/2530] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:42:32 2022] DELETE /identity/v3/users/3dae32fa077840d9a053b6bef162615e => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:33.300931 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482de523-e9c9-47c9-a6db-4bd7be00b155 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:33.303153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-482de523-e9c9-47c9-a6db-4bd7be00b155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:33.316441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482de523-e9c9-47c9-a6db-4bd7be00b155 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:33.376470 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b62841-f89a-4627-a192-040538ce81a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:33.377351 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b62841-f89a-4627-a192-040538ce81a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:33.377632 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b62841-f89a-4627-a192-040538ce81a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:33.377906 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b62841-f89a-4627-a192-040538ce81a0 admin admin] PATH_INFO: `/v3/users/92bfe9f7351c464aa9682e2f39a00788` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:33.379655 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b62841-f89a-4627-a192-040538ce81a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92bfe9f7351c464aa9682e2f39a00788)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:33.381650 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b62841-f89a-4627-a192-040538ce81a0 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:33.517594 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc881bb3-2585-4bfb-8f2f-1ad2c2cfbd68 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:33.519088 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-dc881bb3-2585-4bfb-8f2f-1ad2c2cfbd68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:33.532415 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc881bb3-2585-4bfb-8f2f-1ad2c2cfbd68 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:33.537568 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80004aa-5c70-4f13-802e-ff3b077598b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:33.538460 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80004aa-5c70-4f13-802e-ff3b077598b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:33.538746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80004aa-5c70-4f13-802e-ff3b077598b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:33.539048 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80004aa-5c70-4f13-802e-ff3b077598b5 admin admin] PATH_INFO: `/v3/projects/605d09ab997143aabb55d2733c74116f` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:33.540309 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80004aa-5c70-4f13-802e-ff3b077598b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=605d09ab997143aabb55d2733c74116f)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:33.544644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80004aa-5c70-4f13-802e-ff3b077598b5 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:33.643854 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-97b62841-f89a-4627-a192-040538ce81a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92bfe9f7351c464aa9682e2f39a00788'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:33.675386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a80004aa-5c70-4f13-802e-ff3b077598b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '605d09ab997143aabb55d2733c74116f', 'request_id': 'req-a80004aa-5c70-4f13-802e-ff3b077598b5'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:33.727006 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-97b62841-f89a-4627-a192-040538ce81a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92bfe9f7351c464aa9682e2f39a00788'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:33.739429 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-97b62841-f89a-4627-a192-040538ce81a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92bfe9f7351c464aa9682e2f39a00788'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:33.740062 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-97b62841-f89a-4627-a192-040538ce81a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92bfe9f7351c464aa9682e2f39a00788'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:33.746896 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-97b62841-f89a-4627-a192-040538ce81a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92bfe9f7351c464aa9682e2f39a00788'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:33.752840 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1269/2531] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:42:33 2022] DELETE /identity/v3/users/92bfe9f7351c464aa9682e2f39a00788 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:33.757537 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ced301-db94-4b74-b540-39ab0468d443 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:33.757821 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ced301-db94-4b74-b540-39ab0468d443 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:33.758095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ced301-db94-4b74-b540-39ab0468d443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:33.776433 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-18ced301-db94-4b74-b540-39ab0468d443 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:33.777614 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-18ced301-db94-4b74-b540-39ab0468d443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:33.806379 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a80004aa-5c70-4f13-802e-ff3b077598b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '605d09ab997143aabb55d2733c74116f', 'request_id': 'req-a80004aa-5c70-4f13-802e-ff3b077598b5'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:33.814354 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a80004aa-5c70-4f13-802e-ff3b077598b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '605d09ab997143aabb55d2733c74116f', 'request_id': 'req-a80004aa-5c70-4f13-802e-ff3b077598b5'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:33.821422 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:33.826902 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a80004aa-5c70-4f13-802e-ff3b077598b5 admin admin] The token cache is being invalidate because project 605d09ab997143aabb55d2733c74116f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:42:33.827334 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a80004aa-5c70-4f13-802e-ff3b077598b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:33.828662 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a80004aa-5c70-4f13-802e-ff3b077598b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:33.830225 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1263/2532] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:42:33 2022] DELETE /identity/v3/projects/605d09ab997143aabb55d2733c74116f => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:42:33.837681 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1270/2533] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:42:33.851600 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80004aa-5c70-4f13-802e-ff3b077598b5 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:33.853080 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a80004aa-5c70-4f13-802e-ff3b077598b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:33.916199 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80004aa-5c70-4f13-802e-ff3b077598b5 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:33.920599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:33.970088 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2b2e8-66f0-4fe3-964a-34f8b7a7c2fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:33.970800 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2b2e8-66f0-4fe3-964a-34f8b7a7c2fd service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:33.971099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2b2e8-66f0-4fe3-964a-34f8b7a7c2fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:33.971374 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2b2e8-66f0-4fe3-964a-34f8b7a7c2fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:33.983048 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2b2e8-66f0-4fe3-964a-34f8b7a7c2fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:33.984890 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2b2e8-66f0-4fe3-964a-34f8b7a7c2fd service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:33.995783 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1264/2534] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:42:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:34.476629 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ced301-db94-4b74-b540-39ab0468d443 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:34.478176 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-18ced301-db94-4b74-b540-39ab0468d443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:34.491006 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ced301-db94-4b74-b540-39ab0468d443 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:34.496543 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2f9f7-7169-4c7f-8dd2-5b596c35ee52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:34.497413 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2f9f7-7169-4c7f-8dd2-5b596c35ee52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:34.497696 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2f9f7-7169-4c7f-8dd2-5b596c35ee52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:34.497972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2f9f7-7169-4c7f-8dd2-5b596c35ee52 admin admin] PATH_INFO: `/v3/projects/3bc83f8444c54b109223dea02aa29cce` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:34.499195 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2f9f7-7169-4c7f-8dd2-5b596c35ee52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bc83f8444c54b109223dea02aa29cce)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:34.501101 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2f9f7-7169-4c7f-8dd2-5b596c35ee52 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:34.663881 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-17f2f9f7-7169-4c7f-8dd2-5b596c35ee52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bc83f8444c54b109223dea02aa29cce', 'request_id': 'req-17f2f9f7-7169-4c7f-8dd2-5b596c35ee52'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:34.756031 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-17f2f9f7-7169-4c7f-8dd2-5b596c35ee52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bc83f8444c54b109223dea02aa29cce', 'request_id': 'req-17f2f9f7-7169-4c7f-8dd2-5b596c35ee52'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:34.763546 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-17f2f9f7-7169-4c7f-8dd2-5b596c35ee52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bc83f8444c54b109223dea02aa29cce', 'request_id': 'req-17f2f9f7-7169-4c7f-8dd2-5b596c35ee52'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:34.774723 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-17f2f9f7-7169-4c7f-8dd2-5b596c35ee52 admin admin] The token cache is being invalidate because project 3bc83f8444c54b109223dea02aa29cce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:42:34.775100 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-17f2f9f7-7169-4c7f-8dd2-5b596c35ee52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:34.775804 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-17f2f9f7-7169-4c7f-8dd2-5b596c35ee52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:34.777284 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1271/2535] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:42:34 2022] DELETE /identity/v3/projects/3bc83f8444c54b109223dea02aa29cce => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:34.781362 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2b2e8-66f0-4fe3-964a-34f8b7a7c2fd service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:34.791475 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2b2e8-66f0-4fe3-964a-34f8b7a7c2fd service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:34.846063 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1885ad50-02f7-40c8-bcc9-5c0c604c696d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:34.846950 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885ad50-02f7-40c8-bcc9-5c0c604c696d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:34.847227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885ad50-02f7-40c8-bcc9-5c0c604c696d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:34.847496 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885ad50-02f7-40c8-bcc9-5c0c604c696d admin admin] PATH_INFO: `/v3/users/bcb765bef7b649c29d3a4d64494773a8` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:34.848670 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1885ad50-02f7-40c8-bcc9-5c0c604c696d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcb765bef7b649c29d3a4d64494773a8)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:34.851139 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1885ad50-02f7-40c8-bcc9-5c0c604c696d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:35.112638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1885ad50-02f7-40c8-bcc9-5c0c604c696d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcb765bef7b649c29d3a4d64494773a8'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:35.212688 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1885ad50-02f7-40c8-bcc9-5c0c604c696d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcb765bef7b649c29d3a4d64494773a8'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:35.213310 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1885ad50-02f7-40c8-bcc9-5c0c604c696d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcb765bef7b649c29d3a4d64494773a8'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:35.221346 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1885ad50-02f7-40c8-bcc9-5c0c604c696d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcb765bef7b649c29d3a4d64494773a8'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:35.222054 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1885ad50-02f7-40c8-bcc9-5c0c604c696d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcb765bef7b649c29d3a4d64494773a8'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:35.239615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1265/2536] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:42:34 2022] DELETE /identity/v3/users/bcb765bef7b649c29d3a4d64494773a8 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:35.659184 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2f9f7-7169-4c7f-8dd2-5b596c35ee52 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:35.660597 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-17f2f9f7-7169-4c7f-8dd2-5b596c35ee52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:35.711556 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1885ad50-02f7-40c8-bcc9-5c0c604c696d admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:35.713498 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1885ad50-02f7-40c8-bcc9-5c0c604c696d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:35.719570 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2f9f7-7169-4c7f-8dd2-5b596c35ee52 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:35.752710 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:35.774365 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869da84-dd31-4219-8ff6-de4c04eddc80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:35.775057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869da84-dd31-4219-8ff6-de4c04eddc80 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:35.775340 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869da84-dd31-4219-8ff6-de4c04eddc80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:35.775609 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869da84-dd31-4219-8ff6-de4c04eddc80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:35.777482 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1869da84-dd31-4219-8ff6-de4c04eddc80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:35.797229 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1885ad50-02f7-40c8-bcc9-5c0c604c696d admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:35.802055 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6bf49e-63f2-4f95-bb2c-29d153c1edfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:35.802967 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bf49e-63f2-4f95-bb2c-29d153c1edfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:35.803249 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bf49e-63f2-4f95-bb2c-29d153c1edfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:35.803525 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bf49e-63f2-4f95-bb2c-29d153c1edfb admin admin] PATH_INFO: `/v3/projects/eba5357e4f3a4d32810e063fc530841b` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:35.805263 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6bf49e-63f2-4f95-bb2c-29d153c1edfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=eba5357e4f3a4d32810e063fc530841b)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:35.808664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6bf49e-63f2-4f95-bb2c-29d153c1edfb admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:35.820478 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:35.847148 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869da84-dd31-4219-8ff6-de4c04eddc80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:35.847148 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869da84-dd31-4219-8ff6-de4c04eddc80 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:35.853228 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1272/2537] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:42:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:35.916978 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-db6bf49e-63f2-4f95-bb2c-29d153c1edfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eba5357e4f3a4d32810e063fc530841b', 'request_id': 'req-db6bf49e-63f2-4f95-bb2c-29d153c1edfb'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:35.994968 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-db6bf49e-63f2-4f95-bb2c-29d153c1edfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eba5357e4f3a4d32810e063fc530841b', 'request_id': 'req-db6bf49e-63f2-4f95-bb2c-29d153c1edfb'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:36.001732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-db6bf49e-63f2-4f95-bb2c-29d153c1edfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eba5357e4f3a4d32810e063fc530841b', 'request_id': 'req-db6bf49e-63f2-4f95-bb2c-29d153c1edfb'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:36.012839 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-db6bf49e-63f2-4f95-bb2c-29d153c1edfb admin admin] The token cache is being invalidate because project eba5357e4f3a4d32810e063fc530841b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:42:36.013138 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-db6bf49e-63f2-4f95-bb2c-29d153c1edfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:36.013768 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-db6bf49e-63f2-4f95-bb2c-29d153c1edfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:36.015441 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1266/2538] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:42:35 2022] DELETE /identity/v3/projects/eba5357e4f3a4d32810e063fc530841b => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:36.023735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facafe0a-1108-40f1-8b8e-62e40663e6f2 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:36.023735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facafe0a-1108-40f1-8b8e-62e40663e6f2 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:36.023735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facafe0a-1108-40f1-8b8e-62e40663e6f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:36.039585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-facafe0a-1108-40f1-8b8e-62e40663e6f2 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:36.040800 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-facafe0a-1108-40f1-8b8e-62e40663e6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:36.102809 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1273/2539] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:36.110430 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46a147-46c2-465d-a0b7-5551020e3893 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:36.110717 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46a147-46c2-465d-a0b7-5551020e3893 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:36.111061 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46a147-46c2-465d-a0b7-5551020e3893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:36.131329 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-5a46a147-46c2-465d-a0b7-5551020e3893 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:36.132588 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-5a46a147-46c2-465d-a0b7-5551020e3893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:36.140900 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1267/2540] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:36.145119 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facafe0a-1108-40f1-8b8e-62e40663e6f2 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:36.154919 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facafe0a-1108-40f1-8b8e-62e40663e6f2 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:36.159489 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a62eb0-2f7f-4fd4-8a3a-5eb9708d2518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:36.160143 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a62eb0-2f7f-4fd4-8a3a-5eb9708d2518 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:36.160418 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a62eb0-2f7f-4fd4-8a3a-5eb9708d2518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:36.160687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a62eb0-2f7f-4fd4-8a3a-5eb9708d2518 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:36.161430 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a62eb0-2f7f-4fd4-8a3a-5eb9708d2518 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:42:36.161732 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a62eb0-2f7f-4fd4-8a3a-5eb9708d2518 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:36.163404 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a62eb0-2f7f-4fd4-8a3a-5eb9708d2518 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:36.179234 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1274/2541] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:42:36.180447 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46a147-46c2-465d-a0b7-5551020e3893 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:36.190561 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46a147-46c2-465d-a0b7-5551020e3893 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:36.195966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bfce4f-9f72-4ee1-9da0-cf15c4c546d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:36.196684 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bfce4f-9f72-4ee1-9da0-cf15c4c546d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:36.196968 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bfce4f-9f72-4ee1-9da0-cf15c4c546d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:36.197243 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bfce4f-9f72-4ee1-9da0-cf15c4c546d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:36.198002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bfce4f-9f72-4ee1-9da0-cf15c4c546d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:36.199966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bfce4f-9f72-4ee1-9da0-cf15c4c546d1 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:36.302578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1268/2542] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:42:36 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:42:36.305824 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a62eb0-2f7f-4fd4-8a3a-5eb9708d2518 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:36.316286 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a62eb0-2f7f-4fd4-8a3a-5eb9708d2518 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:36.377383 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b4e0ff-d8f5-4274-b37c-bf7cc0154084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:36.377972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4e0ff-d8f5-4274-b37c-bf7cc0154084 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:36.378252 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4e0ff-d8f5-4274-b37c-bf7cc0154084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:36.378524 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4e0ff-d8f5-4274-b37c-bf7cc0154084 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:36.379243 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b4e0ff-d8f5-4274-b37c-bf7cc0154084 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:36.381205 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b4e0ff-d8f5-4274-b37c-bf7cc0154084 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:36.490933 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1275/2543] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:42:36 2022] POST /identity/v3/users => generated 294 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:42:36.492540 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bfce4f-9f72-4ee1-9da0-cf15c4c546d1 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:36.503207 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bfce4f-9f72-4ee1-9da0-cf15c4c546d1 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:36.507834 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164481d6-ae28-457d-b116-05c5100c6b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:36.508506 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164481d6-ae28-457d-b116-05c5100c6b29 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:36.508782 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164481d6-ae28-457d-b116-05c5100c6b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:36.509058 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164481d6-ae28-457d-b116-05c5100c6b29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:36.509781 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164481d6-ae28-457d-b116-05c5100c6b29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:36.511427 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164481d6-ae28-457d-b116-05c5100c6b29 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:36.523309 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1269/2544] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:42:36.527688 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b4e0ff-d8f5-4274-b37c-bf7cc0154084 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:36.538560 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b4e0ff-d8f5-4274-b37c-bf7cc0154084 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:36.543686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372141e6-c92f-4be7-8e3d-1bbfdae73ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:36.544381 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372141e6-c92f-4be7-8e3d-1bbfdae73ded admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:36.544659 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372141e6-c92f-4be7-8e3d-1bbfdae73ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:36.544932 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372141e6-c92f-4be7-8e3d-1bbfdae73ded admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:36.545649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372141e6-c92f-4be7-8e3d-1bbfdae73ded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:36.547289 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372141e6-c92f-4be7-8e3d-1bbfdae73ded admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:36.561117 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1276/2545] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:42:36.562301 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164481d6-ae28-457d-b116-05c5100c6b29 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:36.572953 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164481d6-ae28-457d-b116-05c5100c6b29 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:36.577532 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b45a4-cf5d-4ee3-a69f-3256c0e1d25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:36.578439 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b45a4-cf5d-4ee3-a69f-3256c0e1d25e admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:36.578718 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b45a4-cf5d-4ee3-a69f-3256c0e1d25e admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:36.579028 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b45a4-cf5d-4ee3-a69f-3256c0e1d25e admin admin] PATH_INFO: `/v3/projects/50431c1b1f1340eea65db98803f1318f/users/70a18412f7fc4da787d5de9a9c6ed093/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:36.594288 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715b45a4-cf5d-4ee3-a69f-3256c0e1d25e admin admin] RBAC: Authorizing `identity:create_grant(project_id=50431c1b1f1340eea65db98803f1318f, user_id=70a18412f7fc4da787d5de9a9c6ed093, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:36.597000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715b45a4-cf5d-4ee3-a69f-3256c0e1d25e admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:36.692667 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1270/2546] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:42:36 2022] PUT /identity/v3/projects/50431c1b1f1340eea65db98803f1318f/users/70a18412f7fc4da787d5de9a9c6ed093/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:36.709819 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a70580-dc2c-4541-a679-5ebe0252c0c0 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:36.710381 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a70580-dc2c-4541-a679-5ebe0252c0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:36.710381 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a70580-dc2c-4541-a679-5ebe0252c0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:36.741921 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-e1a70580-dc2c-4541-a679-5ebe0252c0c0 None None] MFA Rules not processed for user `70a18412f7fc4da787d5de9a9c6ed093`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:36.743234 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e1a70580-dc2c-4541-a679-5ebe0252c0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:36.781546 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:36.824532 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1277/2547] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:36.828975 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b45a4-cf5d-4ee3-a69f-3256c0e1d25e admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:36.896474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:36.902520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b45a4-cf5d-4ee3-a69f-3256c0e1d25e admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:36.908487 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e054e-7b59-43ab-b4d6-46435c813c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:36.909187 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e054e-7b59-43ab-b4d6-46435c813c1c admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:36.909512 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e054e-7b59-43ab-b4d6-46435c813c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:36.909788 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e054e-7b59-43ab-b4d6-46435c813c1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:36.910456 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e054e-7b59-43ab-b4d6-46435c813c1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:36.913408 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e054e-7b59-43ab-b4d6-46435c813c1c admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:37.002510 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1271/2548] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:42:36 2022] POST /identity/v3/projects => generated 387 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:42:37.006963 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a70580-dc2c-4541-a679-5ebe0252c0c0 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:37.018153 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a70580-dc2c-4541-a679-5ebe0252c0c0 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:37.082050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906103b-6c07-44a4-84b8-00581a2f47e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:37.086619 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906103b-6c07-44a4-84b8-00581a2f47e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:37.086954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906103b-6c07-44a4-84b8-00581a2f47e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:37.087223 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906103b-6c07-44a4-84b8-00581a2f47e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:37.087937 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906103b-6c07-44a4-84b8-00581a2f47e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:37.090037 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906103b-6c07-44a4-84b8-00581a2f47e0 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:37.186487 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1278/2549] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:42:37 2022] POST /identity/v3/users => generated 300 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:42:37.187817 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e054e-7b59-43ab-b4d6-46435c813c1c admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:37.205933 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e054e-7b59-43ab-b4d6-46435c813c1c admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:37.215563 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ac26b-8b85-4dff-aad4-b8240740a398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:37.216705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ac26b-8b85-4dff-aad4-b8240740a398 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:37.217220 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ac26b-8b85-4dff-aad4-b8240740a398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:37.217719 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ac26b-8b85-4dff-aad4-b8240740a398 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:37.219058 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5ac26b-8b85-4dff-aad4-b8240740a398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:37.222306 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5ac26b-8b85-4dff-aad4-b8240740a398 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:37.241787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1272/2550] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:42:37.246735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906103b-6c07-44a4-84b8-00581a2f47e0 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:37.257886 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906103b-6c07-44a4-84b8-00581a2f47e0 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:37.262453 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9181cc-df39-4e05-a7e0-97ae0c00a0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:37.263399 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9181cc-df39-4e05-a7e0-97ae0c00a0d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:37.263684 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9181cc-df39-4e05-a7e0-97ae0c00a0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:37.263988 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9181cc-df39-4e05-a7e0-97ae0c00a0d6 admin admin] PATH_INFO: `/v3/projects/49087ca6d42f4a8fbb1dda2ea9dc16ce/users/567dd922f00a4bcf9e4c9b3e74c978dc/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:37.282578 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9181cc-df39-4e05-a7e0-97ae0c00a0d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49087ca6d42f4a8fbb1dda2ea9dc16ce, user_id=567dd922f00a4bcf9e4c9b3e74c978dc, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:37.285279 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9181cc-df39-4e05-a7e0-97ae0c00a0d6 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:37.361419 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1279/2551] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:42:37 2022] PUT /identity/v3/projects/49087ca6d42f4a8fbb1dda2ea9dc16ce/users/567dd922f00a4bcf9e4c9b3e74c978dc/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:37.365984 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ac26b-8b85-4dff-aad4-b8240740a398 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:37.376664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ac26b-8b85-4dff-aad4-b8240740a398 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:37.440262 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0656e8-6c57-4769-969b-71057e820359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:37.440935 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0656e8-6c57-4769-969b-71057e820359 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:37.441220 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0656e8-6c57-4769-969b-71057e820359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:37.441498 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0656e8-6c57-4769-969b-71057e820359 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:37.442227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0656e8-6c57-4769-969b-71057e820359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:37.444624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0656e8-6c57-4769-969b-71057e820359 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:37.459054 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1273/2552] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:42:37.460457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9181cc-df39-4e05-a7e0-97ae0c00a0d6 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:37.471894 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9181cc-df39-4e05-a7e0-97ae0c00a0d6 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:37.477564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422f9eb-9a68-485f-92c6-be853cd503cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:37.478475 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422f9eb-9a68-485f-92c6-be853cd503cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:37.478759 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422f9eb-9a68-485f-92c6-be853cd503cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:37.479069 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422f9eb-9a68-485f-92c6-be853cd503cb admin admin] PATH_INFO: `/v3/projects/49087ca6d42f4a8fbb1dda2ea9dc16ce/users/567dd922f00a4bcf9e4c9b3e74c978dc/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:37.481164 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4422f9eb-9a68-485f-92c6-be853cd503cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=49087ca6d42f4a8fbb1dda2ea9dc16ce, user_id=567dd922f00a4bcf9e4c9b3e74c978dc, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:37.483944 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4422f9eb-9a68-485f-92c6-be853cd503cb admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:37.495265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1280/2553] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:42:37 2022] PUT /identity/v3/projects/49087ca6d42f4a8fbb1dda2ea9dc16ce/users/567dd922f00a4bcf9e4c9b3e74c978dc/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:37.506283 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633c153-fe65-4d7c-936d-433653e4155b None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:37.506600 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633c153-fe65-4d7c-936d-433653e4155b None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:37.507011 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633c153-fe65-4d7c-936d-433653e4155b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:37.541065 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-3633c153-fe65-4d7c-936d-433653e4155b None None] MFA Rules not processed for user `567dd922f00a4bcf9e4c9b3e74c978dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:37.543042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3633c153-fe65-4d7c-936d-433653e4155b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:37.623275 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1274/2554] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:37.634575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422f9eb-9a68-485f-92c6-be853cd503cb admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:37.637795 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4422f9eb-9a68-485f-92c6-be853cd503cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:37.703879 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422f9eb-9a68-485f-92c6-be853cd503cb admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:37.711166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2012b-4405-4552-a1df-ab46c77d271a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:37.711795 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2012b-4405-4552-a1df-ab46c77d271a service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:37.712022 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2012b-4405-4552-a1df-ab46c77d271a service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:37.712294 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2012b-4405-4552-a1df-ab46c77d271a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:37.722947 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2012b-4405-4552-a1df-ab46c77d271a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:37.725054 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2012b-4405-4552-a1df-ab46c77d271a service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:37.735875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1281/2555] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:42:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:37.802555 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633c153-fe65-4d7c-936d-433653e4155b None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:37.854863 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:37.867266 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633c153-fe65-4d7c-936d-433653e4155b None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:37.872003 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb1e168-23a2-4f91-a408-2fa2d35e8896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:37.873024 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1e168-23a2-4f91-a408-2fa2d35e8896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:37.873529 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1e168-23a2-4f91-a408-2fa2d35e8896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:37.873865 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1e168-23a2-4f91-a408-2fa2d35e8896 admin admin] PATH_INFO: `/v3/users/70a18412f7fc4da787d5de9a9c6ed093` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:37.875347 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb1e168-23a2-4f91-a408-2fa2d35e8896 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70a18412f7fc4da787d5de9a9c6ed093)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:37.877406 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb1e168-23a2-4f91-a408-2fa2d35e8896 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:38.101502 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-2bb1e168-23a2-4f91-a408-2fa2d35e8896 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70a18412f7fc4da787d5de9a9c6ed093'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:38.241788 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-2bb1e168-23a2-4f91-a408-2fa2d35e8896 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70a18412f7fc4da787d5de9a9c6ed093'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:38.242370 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-2bb1e168-23a2-4f91-a408-2fa2d35e8896 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70a18412f7fc4da787d5de9a9c6ed093'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:38.249946 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-2bb1e168-23a2-4f91-a408-2fa2d35e8896 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70a18412f7fc4da787d5de9a9c6ed093'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:38.250581 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-2bb1e168-23a2-4f91-a408-2fa2d35e8896 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70a18412f7fc4da787d5de9a9c6ed093'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:38.267987 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1275/2556] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:42:37 2022] DELETE /identity/v3/users/70a18412f7fc4da787d5de9a9c6ed093 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:38.272634 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaab04-aed1-4726-ac2b-a3c783f9e3ec None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:38.272927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaab04-aed1-4726-ac2b-a3c783f9e3ec None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:38.273232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daaab04-aed1-4726-ac2b-a3c783f9e3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:38.290794 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-7daaab04-aed1-4726-ac2b-a3c783f9e3ec None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:38.292021 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-7daaab04-aed1-4726-ac2b-a3c783f9e3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:38.315706 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:38.352049 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1282/2557] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:38.361899 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb1e168-23a2-4f91-a408-2fa2d35e8896 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:38.363341 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-2bb1e168-23a2-4f91-a408-2fa2d35e8896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:38.424073 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb1e168-23a2-4f91-a408-2fa2d35e8896 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:38.428341 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6cec9c-0f9b-4d16-b7ff-9771321e47e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:38.429031 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cec9c-0f9b-4d16-b7ff-9771321e47e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:38.429267 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cec9c-0f9b-4d16-b7ff-9771321e47e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:38.429540 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cec9c-0f9b-4d16-b7ff-9771321e47e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:38.439025 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6cec9c-0f9b-4d16-b7ff-9771321e47e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:38.440453 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6cec9c-0f9b-4d16-b7ff-9771321e47e4 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:38.450726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1276/2558] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:42:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:39.054785 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daaab04-aed1-4726-ac2b-a3c783f9e3ec None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:39.056427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-7daaab04-aed1-4726-ac2b-a3c783f9e3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:39.069360 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daaab04-aed1-4726-ac2b-a3c783f9e3ec None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:39.081322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3a442-ed0d-4656-8641-44545b0d1fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:39.082484 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3a442-ed0d-4656-8641-44545b0d1fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:39.082772 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3a442-ed0d-4656-8641-44545b0d1fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:39.083993 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3a442-ed0d-4656-8641-44545b0d1fce admin admin] PATH_INFO: `/v3/projects/50431c1b1f1340eea65db98803f1318f` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:39.087937 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3a442-ed0d-4656-8641-44545b0d1fce admin admin] RBAC: Authorizing `identity:delete_project(project_id=50431c1b1f1340eea65db98803f1318f)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:39.090477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3a442-ed0d-4656-8641-44545b0d1fce admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:39.252093 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3da3a442-ed0d-4656-8641-44545b0d1fce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50431c1b1f1340eea65db98803f1318f', 'request_id': 'req-3da3a442-ed0d-4656-8641-44545b0d1fce'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:39.348903 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3da3a442-ed0d-4656-8641-44545b0d1fce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50431c1b1f1340eea65db98803f1318f', 'request_id': 'req-3da3a442-ed0d-4656-8641-44545b0d1fce'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:39.356423 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3da3a442-ed0d-4656-8641-44545b0d1fce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50431c1b1f1340eea65db98803f1318f', 'request_id': 'req-3da3a442-ed0d-4656-8641-44545b0d1fce'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:39.367966 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3da3a442-ed0d-4656-8641-44545b0d1fce admin admin] The token cache is being invalidate because project 50431c1b1f1340eea65db98803f1318f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:42:39.368247 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3da3a442-ed0d-4656-8641-44545b0d1fce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:39.369026 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-3da3a442-ed0d-4656-8641-44545b0d1fce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:39.370807 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1283/2559] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:42:39 2022] DELETE /identity/v3/projects/50431c1b1f1340eea65db98803f1318f => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:39.374595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6cec9c-0f9b-4d16-b7ff-9771321e47e4 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:39.392251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6cec9c-0f9b-4d16-b7ff-9771321e47e4 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:39.452851 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5fc61e-9b43-4b6d-a868-a8b84b137317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:39.453865 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fc61e-9b43-4b6d-a868-a8b84b137317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:39.454219 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fc61e-9b43-4b6d-a868-a8b84b137317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:39.454565 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5fc61e-9b43-4b6d-a868-a8b84b137317 admin admin] PATH_INFO: `/v3/users/567dd922f00a4bcf9e4c9b3e74c978dc` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:39.455911 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5fc61e-9b43-4b6d-a868-a8b84b137317 admin admin] RBAC: Authorizing `identity:delete_user(user_id=567dd922f00a4bcf9e4c9b3e74c978dc)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:39.457916 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5fc61e-9b43-4b6d-a868-a8b84b137317 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:39.473925 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:39.713286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-8b5fc61e-9b43-4b6d-a868-a8b84b137317 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '567dd922f00a4bcf9e4c9b3e74c978dc'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:39.806800 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-8b5fc61e-9b43-4b6d-a868-a8b84b137317 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '567dd922f00a4bcf9e4c9b3e74c978dc'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:39.807392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-8b5fc61e-9b43-4b6d-a868-a8b84b137317 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '567dd922f00a4bcf9e4c9b3e74c978dc'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:39.814743 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-8b5fc61e-9b43-4b6d-a868-a8b84b137317 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '567dd922f00a4bcf9e4c9b3e74c978dc'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:39.815399 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-8b5fc61e-9b43-4b6d-a868-a8b84b137317 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '567dd922f00a4bcf9e4c9b3e74c978dc'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:39.839123 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1277/2560] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:42:39 2022] DELETE /identity/v3/users/567dd922f00a4bcf9e4c9b3e74c978dc => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:39.850460 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3a442-ed0d-4656-8641-44545b0d1fce admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:39.852645 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-3da3a442-ed0d-4656-8641-44545b0d1fce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:39.866735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3a442-ed0d-4656-8641-44545b0d1fce admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:39.926202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45be0906-e4de-4026-aa92-338413d28fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:39.927642 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be0906-e4de-4026-aa92-338413d28fc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:39.927927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be0906-e4de-4026-aa92-338413d28fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:39.928201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be0906-e4de-4026-aa92-338413d28fc2 admin admin] PATH_INFO: `/v3/users/104a135932a144269f1f2aa7359098ed` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:39.929458 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45be0906-e4de-4026-aa92-338413d28fc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=104a135932a144269f1f2aa7359098ed)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:39.931607 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45be0906-e4de-4026-aa92-338413d28fc2 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:40.158158 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-45be0906-e4de-4026-aa92-338413d28fc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '104a135932a144269f1f2aa7359098ed'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:40.287315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-45be0906-e4de-4026-aa92-338413d28fc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '104a135932a144269f1f2aa7359098ed'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:40.288866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:40.300171 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-45be0906-e4de-4026-aa92-338413d28fc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '104a135932a144269f1f2aa7359098ed'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:40.300859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-45be0906-e4de-4026-aa92-338413d28fc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '104a135932a144269f1f2aa7359098ed'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:40.308142 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-45be0906-e4de-4026-aa92-338413d28fc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '104a135932a144269f1f2aa7359098ed'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:40.314091 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1284/2561] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:42:39 2022] DELETE /identity/v3/users/104a135932a144269f1f2aa7359098ed => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:42:40.332435 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5fc61e-9b43-4b6d-a868-a8b84b137317 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:40.334848 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-8b5fc61e-9b43-4b6d-a868-a8b84b137317 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:40.347920 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5fc61e-9b43-4b6d-a868-a8b84b137317 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:40.410038 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd719b-5f71-4270-9666-8cc5fad88860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:40.410934 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd719b-5f71-4270-9666-8cc5fad88860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:40.411223 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd719b-5f71-4270-9666-8cc5fad88860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:40.412052 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd719b-5f71-4270-9666-8cc5fad88860 admin admin] PATH_INFO: `/v3/projects/49087ca6d42f4a8fbb1dda2ea9dc16ce` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:40.415087 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffd719b-5f71-4270-9666-8cc5fad88860 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49087ca6d42f4a8fbb1dda2ea9dc16ce)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:40.415455 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffd719b-5f71-4270-9666-8cc5fad88860 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:40.533654 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6ffd719b-5f71-4270-9666-8cc5fad88860 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49087ca6d42f4a8fbb1dda2ea9dc16ce', 'request_id': 'req-6ffd719b-5f71-4270-9666-8cc5fad88860'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:40.620760 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45be0906-e4de-4026-aa92-338413d28fc2 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:40.622257 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-45be0906-e4de-4026-aa92-338413d28fc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:40.635923 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45be0906-e4de-4026-aa92-338413d28fc2 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:40.650183 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6ffd719b-5f71-4270-9666-8cc5fad88860 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49087ca6d42f4a8fbb1dda2ea9dc16ce', 'request_id': 'req-6ffd719b-5f71-4270-9666-8cc5fad88860'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:40.657643 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6ffd719b-5f71-4270-9666-8cc5fad88860 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49087ca6d42f4a8fbb1dda2ea9dc16ce', 'request_id': 'req-6ffd719b-5f71-4270-9666-8cc5fad88860'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:40.668720 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6ffd719b-5f71-4270-9666-8cc5fad88860 admin admin] The token cache is being invalidate because project 49087ca6d42f4a8fbb1dda2ea9dc16ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:42:40.669010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6ffd719b-5f71-4270-9666-8cc5fad88860 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:40.669663 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6ffd719b-5f71-4270-9666-8cc5fad88860 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:40.671266 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1278/2562] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:42:40 2022] DELETE /identity/v3/projects/49087ca6d42f4a8fbb1dda2ea9dc16ce => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:40.675874 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2dc33-7048-4127-8396-aa9c04de13be None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:40.676157 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2dc33-7048-4127-8396-aa9c04de13be None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:40.676431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2dc33-7048-4127-8396-aa9c04de13be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:40.694851 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-61a2dc33-7048-4127-8396-aa9c04de13be None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:40.696187 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-61a2dc33-7048-4127-8396-aa9c04de13be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:40.698606 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093280b4-fb88-4b7c-ab43-b5feddebf207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:40.699495 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093280b4-fb88-4b7c-ab43-b5feddebf207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:40.699771 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093280b4-fb88-4b7c-ab43-b5feddebf207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:40.700038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093280b4-fb88-4b7c-ab43-b5feddebf207 admin admin] PATH_INFO: `/v3/projects/1377b9e588bd44f3b29b03728c418151` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:40.701200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093280b4-fb88-4b7c-ab43-b5feddebf207 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1377b9e588bd44f3b29b03728c418151)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:40.703157 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093280b4-fb88-4b7c-ab43-b5feddebf207 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:40.704616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1279/2563] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:42:40.712943 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46bd542-3525-464d-8c78-fce264852795 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:40.713232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46bd542-3525-464d-8c78-fce264852795 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:40.713514 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46bd542-3525-464d-8c78-fce264852795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:40.731595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-f46bd542-3525-464d-8c78-fce264852795 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:40.732788 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f46bd542-3525-464d-8c78-fce264852795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:40.740818 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1280/2564] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:42:40.745000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46bd542-3525-464d-8c78-fce264852795 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:40.753807 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46bd542-3525-464d-8c78-fce264852795 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:40.759108 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d798e0-ade2-45e8-87ef-9e2b0cfe96fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:40.759775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d798e0-ade2-45e8-87ef-9e2b0cfe96fb admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:40.760058 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d798e0-ade2-45e8-87ef-9e2b0cfe96fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:40.764151 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d798e0-ade2-45e8-87ef-9e2b0cfe96fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:40.764151 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d798e0-ade2-45e8-87ef-9e2b0cfe96fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:42:40.764151 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d798e0-ade2-45e8-87ef-9e2b0cfe96fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:40.764151 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d798e0-ade2-45e8-87ef-9e2b0cfe96fb admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:40.771977 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:40.777646 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1281/2565] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:42:40.779628 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d798e0-ade2-45e8-87ef-9e2b0cfe96fb admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:40.789113 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d798e0-ade2-45e8-87ef-9e2b0cfe96fb admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:40.793660 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31682f01-4fd8-4611-8e57-6bb81b7f0e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:40.794269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31682f01-4fd8-4611-8e57-6bb81b7f0e02 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:40.794548 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31682f01-4fd8-4611-8e57-6bb81b7f0e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:40.794854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31682f01-4fd8-4611-8e57-6bb81b7f0e02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:40.795566 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31682f01-4fd8-4611-8e57-6bb81b7f0e02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:40.797420 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31682f01-4fd8-4611-8e57-6bb81b7f0e02 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:40.852286 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-093280b4-fb88-4b7c-ab43-b5feddebf207 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1377b9e588bd44f3b29b03728c418151', 'request_id': 'req-093280b4-fb88-4b7c-ab43-b5feddebf207'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:40.899867 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1282/2566] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:42:40 2022] POST /identity/v3/projects => generated 401 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:42:40.904100 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31682f01-4fd8-4611-8e57-6bb81b7f0e02 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:40.927595 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-093280b4-fb88-4b7c-ab43-b5feddebf207 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1377b9e588bd44f3b29b03728c418151', 'request_id': 'req-093280b4-fb88-4b7c-ab43-b5feddebf207'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:40.934440 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-093280b4-fb88-4b7c-ab43-b5feddebf207 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1377b9e588bd44f3b29b03728c418151', 'request_id': 'req-093280b4-fb88-4b7c-ab43-b5feddebf207'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:40.946646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-093280b4-fb88-4b7c-ab43-b5feddebf207 admin admin] The token cache is being invalidate because project 1377b9e588bd44f3b29b03728c418151 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:42:40.947055 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-093280b4-fb88-4b7c-ab43-b5feddebf207 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:40.947776 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-093280b4-fb88-4b7c-ab43-b5feddebf207 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:40.949532 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1285/2567] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:42:40 2022] DELETE /identity/v3/projects/1377b9e588bd44f3b29b03728c418151 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:42:40.954362 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7c321-983d-409d-98e9-7a30baeddbdc None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:40.954989 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7c321-983d-409d-98e9-7a30baeddbdc None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:40.954989 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7c321-983d-409d-98e9-7a30baeddbdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:40.966191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31682f01-4fd8-4611-8e57-6bb81b7f0e02 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:40.972217 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1690a2b-c13b-4f55-ba1f-3434d6b88880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:40.973102 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1690a2b-c13b-4f55-ba1f-3434d6b88880 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:40.973388 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1690a2b-c13b-4f55-ba1f-3434d6b88880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:40.973794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1690a2b-c13b-4f55-ba1f-3434d6b88880 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:40.974482 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1690a2b-c13b-4f55-ba1f-3434d6b88880 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:40.974613 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-5bf7c321-983d-409d-98e9-7a30baeddbdc None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:40.975819 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5bf7c321-983d-409d-98e9-7a30baeddbdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:40.976519 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1690a2b-c13b-4f55-ba1f-3434d6b88880 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:40.984430 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1286/2568] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:42:40.992051 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407270b-7b16-4f15-b31e-9e87b1b0d24b None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:40.992341 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407270b-7b16-4f15-b31e-9e87b1b0d24b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:40.992624 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407270b-7b16-4f15-b31e-9e87b1b0d24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:41.011389 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-7407270b-7b16-4f15-b31e-9e87b1b0d24b None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:41.012599 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-7407270b-7b16-4f15-b31e-9e87b1b0d24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:41.021585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1287/2569] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:42:41.025667 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7407270b-7b16-4f15-b31e-9e87b1b0d24b None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:41.034894 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7407270b-7b16-4f15-b31e-9e87b1b0d24b None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:41.039663 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495c2d0-74a1-468e-afd9-df3977f28ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:41.040310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495c2d0-74a1-468e-afd9-df3977f28ece admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:41.040597 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495c2d0-74a1-468e-afd9-df3977f28ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:41.040875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495c2d0-74a1-468e-afd9-df3977f28ece admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:41.041600 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495c2d0-74a1-468e-afd9-df3977f28ece admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:42:41.041912 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495c2d0-74a1-468e-afd9-df3977f28ece admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:41.043572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495c2d0-74a1-468e-afd9-df3977f28ece admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:41.059034 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1288/2570] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:42:41.059991 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495c2d0-74a1-468e-afd9-df3977f28ece admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:41.070106 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495c2d0-74a1-468e-afd9-df3977f28ece admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:41.079779 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18212b77-202b-4cbb-b72f-00ae60323cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:41.080448 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18212b77-202b-4cbb-b72f-00ae60323cd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:41.080735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18212b77-202b-4cbb-b72f-00ae60323cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:41.081536 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18212b77-202b-4cbb-b72f-00ae60323cd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:41.081697 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18212b77-202b-4cbb-b72f-00ae60323cd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:41.084574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18212b77-202b-4cbb-b72f-00ae60323cd6 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:41.098348 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1283/2571] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:42:40 2022] POST /identity/v3/users => generated 301 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:42:41.102678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1690a2b-c13b-4f55-ba1f-3434d6b88880 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:41.104292 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f1690a2b-c13b-4f55-ba1f-3434d6b88880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:41.116618 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1690a2b-c13b-4f55-ba1f-3434d6b88880 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:41.121498 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851298b4-38ba-423c-9732-886b7394c15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:41.122176 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851298b4-38ba-423c-9732-886b7394c15a admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:41.122461 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851298b4-38ba-423c-9732-886b7394c15a admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:41.122738 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851298b4-38ba-423c-9732-886b7394c15a admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:41.123518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851298b4-38ba-423c-9732-886b7394c15a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:41.125136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851298b4-38ba-423c-9732-886b7394c15a admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:41.135748 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1284/2572] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:42:41.144306 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851298b4-38ba-423c-9732-886b7394c15a admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:41.154213 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851298b4-38ba-423c-9732-886b7394c15a admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:41.158930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c071f92-ce56-4467-9a7c-1a7346e42310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:41.159622 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c071f92-ce56-4467-9a7c-1a7346e42310 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:41.159902 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c071f92-ce56-4467-9a7c-1a7346e42310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:41.160179 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c071f92-ce56-4467-9a7c-1a7346e42310 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:41.160897 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c071f92-ce56-4467-9a7c-1a7346e42310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:41.162523 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c071f92-ce56-4467-9a7c-1a7346e42310 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:41.172853 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1285/2573] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:42:41.176791 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c071f92-ce56-4467-9a7c-1a7346e42310 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:41.186717 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c071f92-ce56-4467-9a7c-1a7346e42310 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:41.191447 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6889af7-4c08-40b0-9cf6-97b2669f2d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:41.192293 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6889af7-4c08-40b0-9cf6-97b2669f2d83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:41.192651 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6889af7-4c08-40b0-9cf6-97b2669f2d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:41.193431 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6889af7-4c08-40b0-9cf6-97b2669f2d83 admin admin] PATH_INFO: `/v3/projects/a7dbf8f3705a4b519170465ad13d9b0e/users/dbf3f8ef7c0b4c32a2b69044ad3235e7/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:41.202708 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1289/2574] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:42:41 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:42:41.206744 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18212b77-202b-4cbb-b72f-00ae60323cd6 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:41.209713 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6889af7-4c08-40b0-9cf6-97b2669f2d83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7dbf8f3705a4b519170465ad13d9b0e, user_id=dbf3f8ef7c0b4c32a2b69044ad3235e7, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:41.212403 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6889af7-4c08-40b0-9cf6-97b2669f2d83 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:41.220060 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:41.269808 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18212b77-202b-4cbb-b72f-00ae60323cd6 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:41.274509 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76bdbd9-b23c-444a-8aa2-dc0b390f613d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:41.275197 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bdbd9-b23c-444a-8aa2-dc0b390f613d admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:41.275480 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bdbd9-b23c-444a-8aa2-dc0b390f613d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:41.275754 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bdbd9-b23c-444a-8aa2-dc0b390f613d admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:41.276441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76bdbd9-b23c-444a-8aa2-dc0b390f613d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:41.278359 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76bdbd9-b23c-444a-8aa2-dc0b390f613d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:41.315898 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1286/2575] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:42:41 2022] PUT /identity/v3/projects/a7dbf8f3705a4b519170465ad13d9b0e/users/dbf3f8ef7c0b4c32a2b69044ad3235e7/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:42:41.329195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0963a-b0fa-4e37-bb06-b415ac262e9f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:41.329650 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0963a-b0fa-4e37-bb06-b415ac262e9f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:41.329931 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0963a-b0fa-4e37-bb06-b415ac262e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:41.355558 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-9ba0963a-b0fa-4e37-bb06-b415ac262e9f None None] MFA Rules not processed for user `dbf3f8ef7c0b4c32a2b69044ad3235e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:41.356773 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-9ba0963a-b0fa-4e37-bb06-b415ac262e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:41.412955 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1290/2576] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:42:41 2022] POST /identity/v3/users => generated 292 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:42:41.416986 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76bdbd9-b23c-444a-8aa2-dc0b390f613d admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:41.419825 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1287/2577] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:42:41.424047 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0963a-b0fa-4e37-bb06-b415ac262e9f None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:41.426821 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76bdbd9-b23c-444a-8aa2-dc0b390f613d admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:41.486950 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6c6fa-8127-4739-8c41-2b29f5516346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:41.487405 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0963a-b0fa-4e37-bb06-b415ac262e9f None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:41.487629 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6c6fa-8127-4739-8c41-2b29f5516346 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:41.487910 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6c6fa-8127-4739-8c41-2b29f5516346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:41.488185 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6c6fa-8127-4739-8c41-2b29f5516346 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:41.488912 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6c6fa-8127-4739-8c41-2b29f5516346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:41.491140 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6c6fa-8127-4739-8c41-2b29f5516346 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:41.492559 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48959993-abf4-4c69-b12d-ef52d849010e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:41.493220 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48959993-abf4-4c69-b12d-ef52d849010e admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:41.493497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48959993-abf4-4c69-b12d-ef52d849010e admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:41.493776 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48959993-abf4-4c69-b12d-ef52d849010e admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:41.494455 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48959993-abf4-4c69-b12d-ef52d849010e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:41.496553 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48959993-abf4-4c69-b12d-ef52d849010e admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:41.503843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1291/2578] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:42:41.508486 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6c6fa-8127-4739-8c41-2b29f5516346 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:41.518314 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6c6fa-8127-4739-8c41-2b29f5516346 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:41.523547 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8dfd2e-2be4-43e5-b247-560b2858efb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:41.524199 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8dfd2e-2be4-43e5-b247-560b2858efb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:41.524484 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8dfd2e-2be4-43e5-b247-560b2858efb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:41.524760 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8dfd2e-2be4-43e5-b247-560b2858efb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:41.525528 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8dfd2e-2be4-43e5-b247-560b2858efb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:41.527672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8dfd2e-2be4-43e5-b247-560b2858efb2 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:41.538201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1292/2579] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:42:41.542270 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8dfd2e-2be4-43e5-b247-560b2858efb2 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:41.551353 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8dfd2e-2be4-43e5-b247-560b2858efb2 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:41.556014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a04a2-a1e9-47a9-a95f-37080e358f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:41.556858 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a04a2-a1e9-47a9-a95f-37080e358f8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:41.557139 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a04a2-a1e9-47a9-a95f-37080e358f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:41.557413 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a04a2-a1e9-47a9-a95f-37080e358f8d admin admin] PATH_INFO: `/v3/projects/a51d843f78a54e8b9dbba4ab9b28accf/users/998a4598a29849908051773834fe55ad/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:41.572242 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a04a2-a1e9-47a9-a95f-37080e358f8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a51d843f78a54e8b9dbba4ab9b28accf, user_id=998a4598a29849908051773834fe55ad, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:41.574902 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1a04a2-a1e9-47a9-a95f-37080e358f8d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:41.608386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:41.615901 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1288/2580] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:42:41 2022] POST /identity/v3/projects => generated 401 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:42:41.619613 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48959993-abf4-4c69-b12d-ef52d849010e admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:41.680592 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1293/2581] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:42:41 2022] PUT /identity/v3/projects/a51d843f78a54e8b9dbba4ab9b28accf/users/998a4598a29849908051773834fe55ad/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:42:41.680865 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48959993-abf4-4c69-b12d-ef52d849010e admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:41.691147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc75e1c-da09-4203-96ae-27322d0e98f2 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:41.691459 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc75e1c-da09-4203-96ae-27322d0e98f2 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:41.691719 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc75e1c-da09-4203-96ae-27322d0e98f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:41.719204 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-acc75e1c-da09-4203-96ae-27322d0e98f2 None None] MFA Rules not processed for user `998a4598a29849908051773834fe55ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:41.720451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-acc75e1c-da09-4203-96ae-27322d0e98f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:41.735851 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7940e0-1ec2-47a9-a6e2-fb0745374cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:41.736563 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7940e0-1ec2-47a9-a6e2-fb0745374cb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:41.736834 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7940e0-1ec2-47a9-a6e2-fb0745374cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:41.737142 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7940e0-1ec2-47a9-a6e2-fb0745374cb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:41.737879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7940e0-1ec2-47a9-a6e2-fb0745374cb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:41.739898 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7940e0-1ec2-47a9-a6e2-fb0745374cb4 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:41.788273 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1294/2582] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:42:41.799786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc75e1c-da09-4203-96ae-27322d0e98f2 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:41.801183 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-acc75e1c-da09-4203-96ae-27322d0e98f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:41.850599 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1289/2583] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:42:41 2022] POST /identity/v3/users => generated 307 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:42:41.852253 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7940e0-1ec2-47a9-a6e2-fb0745374cb4 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:41.863050 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7940e0-1ec2-47a9-a6e2-fb0745374cb4 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:41.869123 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5d5b7-3c88-4e3e-8f27-5b3774d27ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:41.870035 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5d5b7-3c88-4e3e-8f27-5b3774d27ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:41.870168 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5d5b7-3c88-4e3e-8f27-5b3774d27ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:41.870694 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5d5b7-3c88-4e3e-8f27-5b3774d27ba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:41.871220 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5d5b7-3c88-4e3e-8f27-5b3774d27ba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:41.873017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5d5b7-3c88-4e3e-8f27-5b3774d27ba9 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:41.882202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc75e1c-da09-4203-96ae-27322d0e98f2 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:41.884459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1290/2584] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:42:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:42:41.888013 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c48e5-0d27-4840-a9d9-83602617bb35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:41.888195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5d5b7-3c88-4e3e-8f27-5b3774d27ba9 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:41.888686 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c48e5-0d27-4840-a9d9-83602617bb35 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:41.889019 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c48e5-0d27-4840-a9d9-83602617bb35 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:41.889398 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c48e5-0d27-4840-a9d9-83602617bb35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:41.898031 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c48e5-0d27-4840-a9d9-83602617bb35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:41.899648 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c48e5-0d27-4840-a9d9-83602617bb35 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:41.900148 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5d5b7-3c88-4e3e-8f27-5b3774d27ba9 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:41.905762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639bc34-e496-4c42-b760-19a6b08fe36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:41.906828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639bc34-e496-4c42-b760-19a6b08fe36e admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:41.907189 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639bc34-e496-4c42-b760-19a6b08fe36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:41.907543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639bc34-e496-4c42-b760-19a6b08fe36e admin admin] PATH_INFO: `/v3/projects/f09fc77125dd4c33b3b870960a50ed16/users/baa8d721ea1846fe8959d2cb296c071f/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:41.909999 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1295/2585] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:42:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:42:41.927205 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8639bc34-e496-4c42-b760-19a6b08fe36e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f09fc77125dd4c33b3b870960a50ed16, user_id=baa8d721ea1846fe8959d2cb296c071f, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:41.929990 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8639bc34-e496-4c42-b760-19a6b08fe36e admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:42.007064 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:42.092202 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1291/2586] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:42:41 2022] PUT /identity/v3/projects/f09fc77125dd4c33b3b870960a50ed16/users/baa8d721ea1846fe8959d2cb296c071f/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:42:42.096206 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c48e5-0d27-4840-a9d9-83602617bb35 service nova] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:42.107136 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c48e5-0d27-4840-a9d9-83602617bb35 service nova] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:42.162441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e0b30-23e4-472a-a0ca-bc10f4db831d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:42.163056 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e0b30-23e4-472a-a0ca-bc10f4db831d admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:42.163337 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e0b30-23e4-472a-a0ca-bc10f4db831d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:42.163608 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e0b30-23e4-472a-a0ca-bc10f4db831d admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:42.164322 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1e0b30-23e4-472a-a0ca-bc10f4db831d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:42.165937 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1e0b30-23e4-472a-a0ca-bc10f4db831d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:42.175475 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1296/2587] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:42:42.179632 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639bc34-e496-4c42-b760-19a6b08fe36e admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:42.189936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639bc34-e496-4c42-b760-19a6b08fe36e admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:42.194676 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a6547-5743-4381-a01b-49f992683a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:42.195608 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a6547-5743-4381-a01b-49f992683a99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:42.195890 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a6547-5743-4381-a01b-49f992683a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:42.196167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a6547-5743-4381-a01b-49f992683a99 admin admin] PATH_INFO: `/v3/projects/f09fc77125dd4c33b3b870960a50ed16/users/baa8d721ea1846fe8959d2cb296c071f/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:42.198112 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a6547-5743-4381-a01b-49f992683a99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f09fc77125dd4c33b3b870960a50ed16, user_id=baa8d721ea1846fe8959d2cb296c071f, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:42.200954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a6547-5743-4381-a01b-49f992683a99 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:42.211789 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1292/2588] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:42:42 2022] PUT /identity/v3/projects/f09fc77125dd4c33b3b870960a50ed16/users/baa8d721ea1846fe8959d2cb296c071f/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:42.221939 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ead33f-da72-45eb-8c95-afbb9d91e87b None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:42.222797 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ead33f-da72-45eb-8c95-afbb9d91e87b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:42.223105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ead33f-da72-45eb-8c95-afbb9d91e87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:42.250203 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-63ead33f-da72-45eb-8c95-afbb9d91e87b None None] MFA Rules not processed for user `baa8d721ea1846fe8959d2cb296c071f`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:42.251428 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-63ead33f-da72-45eb-8c95-afbb9d91e87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:42.264547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:42.318529 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1297/2589] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:42 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:42.329018 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a6547-5743-4381-a01b-49f992683a99 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:42.330772 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-be2a6547-5743-4381-a01b-49f992683a99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:42.387983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a6547-5743-4381-a01b-49f992683a99 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:42.391613 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6eba60-87b8-4150-86e6-c90b4767fa15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:42.392249 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6eba60-87b8-4150-86e6-c90b4767fa15 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:42.392525 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6eba60-87b8-4150-86e6-c90b4767fa15 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:42.392794 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6eba60-87b8-4150-86e6-c90b4767fa15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:42.401626 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6eba60-87b8-4150-86e6-c90b4767fa15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:42.403080 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6eba60-87b8-4150-86e6-c90b4767fa15 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:42.412756 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1293/2590] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:42:42 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:42.679835 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ead33f-da72-45eb-8c95-afbb9d91e87b None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:42.689481 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ead33f-da72-45eb-8c95-afbb9d91e87b None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:42.693105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca8de9f-1789-4a3a-9207-81e986d35fce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:42.693725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8de9f-1789-4a3a-9207-81e986d35fce service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:42.694009 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8de9f-1789-4a3a-9207-81e986d35fce service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:42.694283 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca8de9f-1789-4a3a-9207-81e986d35fce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:42.749417 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca8de9f-1789-4a3a-9207-81e986d35fce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:42.750867 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca8de9f-1789-4a3a-9207-81e986d35fce service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:42.759821 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1298/2591] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:42:42 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:42.870760 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:49.196815 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6eba60-87b8-4150-86e6-c90b4767fa15 service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:49.259422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6eba60-87b8-4150-86e6-c90b4767fa15 service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:49.264128 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f922eb-b167-4708-bfc9-869f8b57de68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:49.265001 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f922eb-b167-4708-bfc9-869f8b57de68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:49.266319 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f922eb-b167-4708-bfc9-869f8b57de68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:49.266319 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f922eb-b167-4708-bfc9-869f8b57de68 admin admin] PATH_INFO: `/v3/users/dbf3f8ef7c0b4c32a2b69044ad3235e7` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:49.266704 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f922eb-b167-4708-bfc9-869f8b57de68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbf3f8ef7c0b4c32a2b69044ad3235e7)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:49.268664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f922eb-b167-4708-bfc9-869f8b57de68 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:49.483004 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-92f922eb-b167-4708-bfc9-869f8b57de68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbf3f8ef7c0b4c32a2b69044ad3235e7'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:49.548958 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-92f922eb-b167-4708-bfc9-869f8b57de68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbf3f8ef7c0b4c32a2b69044ad3235e7'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:49.549571 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-92f922eb-b167-4708-bfc9-869f8b57de68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbf3f8ef7c0b4c32a2b69044ad3235e7'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:49.558919 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-92f922eb-b167-4708-bfc9-869f8b57de68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbf3f8ef7c0b4c32a2b69044ad3235e7'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:49.559759 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-92f922eb-b167-4708-bfc9-869f8b57de68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbf3f8ef7c0b4c32a2b69044ad3235e7'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:49.585193 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1294/2592] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:42:49 2022] DELETE /identity/v3/users/dbf3f8ef7c0b4c32a2b69044ad3235e7 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:49.589899 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12bef91-77d8-41bd-9439-7f70b1c947f9 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:49.590412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12bef91-77d8-41bd-9439-7f70b1c947f9 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:49.590520 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12bef91-77d8-41bd-9439-7f70b1c947f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:49.608011 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-a12bef91-77d8-41bd-9439-7f70b1c947f9 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:49.609193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a12bef91-77d8-41bd-9439-7f70b1c947f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:49.666556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1299/2593] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:49.677462 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f922eb-b167-4708-bfc9-869f8b57de68 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:49.679039 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-92f922eb-b167-4708-bfc9-869f8b57de68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:49.684631 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:49.738752 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f922eb-b167-4708-bfc9-869f8b57de68 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:49.743002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e7cc2-d4f2-41ce-9d49-1309e3713501 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:49.743653 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e7cc2-d4f2-41ce-9d49-1309e3713501 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:49.743941 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e7cc2-d4f2-41ce-9d49-1309e3713501 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:49.744227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e7cc2-d4f2-41ce-9d49-1309e3713501 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:49.753292 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e7cc2-d4f2-41ce-9d49-1309e3713501 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:49.754736 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e7cc2-d4f2-41ce-9d49-1309e3713501 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:49.764949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1295/2594] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:42:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:50.273218 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12bef91-77d8-41bd-9439-7f70b1c947f9 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:50.273218 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-a12bef91-77d8-41bd-9439-7f70b1c947f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:50.280042 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12bef91-77d8-41bd-9439-7f70b1c947f9 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:50.284709 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5340d197-2b29-431f-92d5-0c351bc7a1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:50.285585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340d197-2b29-431f-92d5-0c351bc7a1b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:50.285868 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340d197-2b29-431f-92d5-0c351bc7a1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:50.286651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340d197-2b29-431f-92d5-0c351bc7a1b6 admin admin] PATH_INFO: `/v3/projects/a7dbf8f3705a4b519170465ad13d9b0e` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:50.287890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5340d197-2b29-431f-92d5-0c351bc7a1b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7dbf8f3705a4b519170465ad13d9b0e)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:50.289831 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5340d197-2b29-431f-92d5-0c351bc7a1b6 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:50.423948 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5340d197-2b29-431f-92d5-0c351bc7a1b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7dbf8f3705a4b519170465ad13d9b0e', 'request_id': 'req-5340d197-2b29-431f-92d5-0c351bc7a1b6'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:50.526550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5340d197-2b29-431f-92d5-0c351bc7a1b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7dbf8f3705a4b519170465ad13d9b0e', 'request_id': 'req-5340d197-2b29-431f-92d5-0c351bc7a1b6'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:50.533072 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5340d197-2b29-431f-92d5-0c351bc7a1b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7dbf8f3705a4b519170465ad13d9b0e', 'request_id': 'req-5340d197-2b29-431f-92d5-0c351bc7a1b6'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:50.545716 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5340d197-2b29-431f-92d5-0c351bc7a1b6 admin admin] The token cache is being invalidate because project a7dbf8f3705a4b519170465ad13d9b0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:42:50.546014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5340d197-2b29-431f-92d5-0c351bc7a1b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:50.546668 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-5340d197-2b29-431f-92d5-0c351bc7a1b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:50.548456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1300/2595] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:42:50 2022] DELETE /identity/v3/projects/a7dbf8f3705a4b519170465ad13d9b0e => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:50.552456 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e7cc2-d4f2-41ce-9d49-1309e3713501 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:50.562152 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e7cc2-d4f2-41ce-9d49-1309e3713501 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:50.619028 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f7676-68a5-432f-ac8d-a97d7ad96ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:50.619909 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f7676-68a5-432f-ac8d-a97d7ad96ffa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:50.620190 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f7676-68a5-432f-ac8d-a97d7ad96ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:50.620468 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710f7676-68a5-432f-ac8d-a97d7ad96ffa admin admin] PATH_INFO: `/v3/users/baa8d721ea1846fe8959d2cb296c071f` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:50.621662 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710f7676-68a5-432f-ac8d-a97d7ad96ffa admin admin] RBAC: Authorizing `identity:delete_user(user_id=baa8d721ea1846fe8959d2cb296c071f)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:50.623637 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710f7676-68a5-432f-ac8d-a97d7ad96ffa admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:50.791159 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-710f7676-68a5-432f-ac8d-a97d7ad96ffa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baa8d721ea1846fe8959d2cb296c071f'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:50.857555 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-710f7676-68a5-432f-ac8d-a97d7ad96ffa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baa8d721ea1846fe8959d2cb296c071f'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:50.858182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-710f7676-68a5-432f-ac8d-a97d7ad96ffa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baa8d721ea1846fe8959d2cb296c071f'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:50.865832 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-710f7676-68a5-432f-ac8d-a97d7ad96ffa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baa8d721ea1846fe8959d2cb296c071f'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:50.866597 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-710f7676-68a5-432f-ac8d-a97d7ad96ffa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baa8d721ea1846fe8959d2cb296c071f'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:50.884595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1296/2596] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:42:50 2022] DELETE /identity/v3/users/baa8d721ea1846fe8959d2cb296c071f => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:42:51.180211 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5340d197-2b29-431f-92d5-0c351bc7a1b6 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:51.180211 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5340d197-2b29-431f-92d5-0c351bc7a1b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:51.193488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5340d197-2b29-431f-92d5-0c351bc7a1b6 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:51.213364 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:51.251048 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced1966-b8be-407c-9f52-5d603050a705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:51.252008 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced1966-b8be-407c-9f52-5d603050a705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:51.252290 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced1966-b8be-407c-9f52-5d603050a705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:51.252562 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced1966-b8be-407c-9f52-5d603050a705 admin admin] PATH_INFO: `/v3/users/998a4598a29849908051773834fe55ad` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:51.253729 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced1966-b8be-407c-9f52-5d603050a705 admin admin] RBAC: Authorizing `identity:delete_user(user_id=998a4598a29849908051773834fe55ad)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:51.256453 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced1966-b8be-407c-9f52-5d603050a705 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:51.351487 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f7676-68a5-432f-ac8d-a97d7ad96ffa admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:51.352939 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-710f7676-68a5-432f-ac8d-a97d7ad96ffa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:51.369248 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710f7676-68a5-432f-ac8d-a97d7ad96ffa admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:51.374485 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a726f04-39f1-4e8f-b0d3-c7810112e944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:51.378251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a726f04-39f1-4e8f-b0d3-c7810112e944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:51.378251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a726f04-39f1-4e8f-b0d3-c7810112e944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:51.378251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a726f04-39f1-4e8f-b0d3-c7810112e944 admin admin] PATH_INFO: `/v3/projects/f09fc77125dd4c33b3b870960a50ed16` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:51.378251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a726f04-39f1-4e8f-b0d3-c7810112e944 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f09fc77125dd4c33b3b870960a50ed16)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:51.379237 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a726f04-39f1-4e8f-b0d3-c7810112e944 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:51.403768 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:51.420448 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-2ced1966-b8be-407c-9f52-5d603050a705 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '998a4598a29849908051773834fe55ad'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:51.476880 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-2ced1966-b8be-407c-9f52-5d603050a705 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '998a4598a29849908051773834fe55ad'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:51.482597 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0a726f04-39f1-4e8f-b0d3-c7810112e944 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f09fc77125dd4c33b3b870960a50ed16', 'request_id': 'req-0a726f04-39f1-4e8f-b0d3-c7810112e944'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:51.489748 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-2ced1966-b8be-407c-9f52-5d603050a705 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '998a4598a29849908051773834fe55ad'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:51.490480 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-2ced1966-b8be-407c-9f52-5d603050a705 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '998a4598a29849908051773834fe55ad'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:51.497268 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-2ced1966-b8be-407c-9f52-5d603050a705 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '998a4598a29849908051773834fe55ad'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:51.503202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1301/2597] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:42:51 2022] DELETE /identity/v3/users/998a4598a29849908051773834fe55ad => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:51.507832 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e9bde-e755-4bd7-ba4b-fc9348960365 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:51.508122 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e9bde-e755-4bd7-ba4b-fc9348960365 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:51.508427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e9bde-e755-4bd7-ba4b-fc9348960365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:51.526103 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-ec2e9bde-e755-4bd7-ba4b-fc9348960365 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:51.527329 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ec2e9bde-e755-4bd7-ba4b-fc9348960365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:51.555018 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0a726f04-39f1-4e8f-b0d3-c7810112e944 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f09fc77125dd4c33b3b870960a50ed16', 'request_id': 'req-0a726f04-39f1-4e8f-b0d3-c7810112e944'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:51.563527 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0a726f04-39f1-4e8f-b0d3-c7810112e944 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f09fc77125dd4c33b3b870960a50ed16', 'request_id': 'req-0a726f04-39f1-4e8f-b0d3-c7810112e944'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:51.579110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0a726f04-39f1-4e8f-b0d3-c7810112e944 admin admin] The token cache is being invalidate because project f09fc77125dd4c33b3b870960a50ed16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:42:51.579673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0a726f04-39f1-4e8f-b0d3-c7810112e944 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:51.580462 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-0a726f04-39f1-4e8f-b0d3-c7810112e944 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:51.581913 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1297/2598] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:42:51 2022] DELETE /identity/v3/projects/f09fc77125dd4c33b3b870960a50ed16 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:42:51.596778 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290be57-0e99-4bc1-8d3d-6699cd7c929c None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:51.597079 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290be57-0e99-4bc1-8d3d-6699cd7c929c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:51.597505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290be57-0e99-4bc1-8d3d-6699cd7c929c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:51.600725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1302/2599] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:42:51.616578 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e9bde-e755-4bd7-ba4b-fc9348960365 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:51.618384 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ec2e9bde-e755-4bd7-ba4b-fc9348960365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:51.623644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-6290be57-0e99-4bc1-8d3d-6699cd7c929c None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:51.625070 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6290be57-0e99-4bc1-8d3d-6699cd7c929c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:51.635040 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1298/2600] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:42:51.644481 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f887780-88b3-41b3-93af-ba567c159d37 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:51.644850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f887780-88b3-41b3-93af-ba567c159d37 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:51.645047 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f887780-88b3-41b3-93af-ba567c159d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:51.668484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-0f887780-88b3-41b3-93af-ba567c159d37 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:51.669711 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-0f887780-88b3-41b3-93af-ba567c159d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:51.680164 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1299/2601] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:42:51.685448 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f887780-88b3-41b3-93af-ba567c159d37 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:51.697173 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e9bde-e755-4bd7-ba4b-fc9348960365 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:51.697406 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f887780-88b3-41b3-93af-ba567c159d37 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:51.702318 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb20c66-4c21-48e7-8f5c-c054d8045834 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:51.705696 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb20c66-4c21-48e7-8f5c-c054d8045834 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:51.705696 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb20c66-4c21-48e7-8f5c-c054d8045834 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:51.705696 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb20c66-4c21-48e7-8f5c-c054d8045834 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:51.705993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac765d-ddff-4552-8537-a1d065722025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:51.705993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac765d-ddff-4552-8537-a1d065722025 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:51.705993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac765d-ddff-4552-8537-a1d065722025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:51.705993 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac765d-ddff-4552-8537-a1d065722025 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:51.706537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac765d-ddff-4552-8537-a1d065722025 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:42:51.707055 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac765d-ddff-4552-8537-a1d065722025 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:51.709493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac765d-ddff-4552-8537-a1d065722025 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:51.714779 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb20c66-4c21-48e7-8f5c-c054d8045834 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:51.716618 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb20c66-4c21-48e7-8f5c-c054d8045834 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:51.728989 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:51.729207 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1300/2602] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:42:51.730704 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac765d-ddff-4552-8537-a1d065722025 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:51.735008 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1303/2603] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:42:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:42:51.741139 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac765d-ddff-4552-8537-a1d065722025 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:51.746294 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0592f-5730-4f42-b647-228680001013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:51.752158 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0592f-5730-4f42-b647-228680001013 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:51.752158 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0592f-5730-4f42-b647-228680001013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:51.752158 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba0592f-5730-4f42-b647-228680001013 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:51.752158 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0592f-5730-4f42-b647-228680001013 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:51.752158 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba0592f-5730-4f42-b647-228680001013 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:51.891892 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1301/2604] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:42:51 2022] POST /identity/v3/projects => generated 387 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:42:51.896906 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb20c66-4c21-48e7-8f5c-c054d8045834 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:51.906719 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb20c66-4c21-48e7-8f5c-c054d8045834 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:51.967953 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00e421d-ba59-4375-b0bb-9a9fb8342569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:51.968632 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e421d-ba59-4375-b0bb-9a9fb8342569 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:51.968911 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e421d-ba59-4375-b0bb-9a9fb8342569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:51.969182 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e421d-ba59-4375-b0bb-9a9fb8342569 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:51.969916 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00e421d-ba59-4375-b0bb-9a9fb8342569 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:51.972019 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00e421d-ba59-4375-b0bb-9a9fb8342569 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:52.057564 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1304/2605] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:42:51 2022] POST /identity/v3/users => generated 294 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:42:52.061369 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0592f-5730-4f42-b647-228680001013 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:52.084727 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba0592f-5730-4f42-b647-228680001013 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:52.090677 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1193f65b-61f5-4a4f-b2ce-813406c2fe8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:52.093812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193f65b-61f5-4a4f-b2ce-813406c2fe8d admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:52.093812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193f65b-61f5-4a4f-b2ce-813406c2fe8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:52.093812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193f65b-61f5-4a4f-b2ce-813406c2fe8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:52.093812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1193f65b-61f5-4a4f-b2ce-813406c2fe8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:52.093812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1193f65b-61f5-4a4f-b2ce-813406c2fe8d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:52.105944 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1302/2606] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:42:52.110232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00e421d-ba59-4375-b0bb-9a9fb8342569 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:52.123946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00e421d-ba59-4375-b0bb-9a9fb8342569 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:52.134037 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699808e7-d566-4f63-81c2-601932ff9fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:52.134714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699808e7-d566-4f63-81c2-601932ff9fbc admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:52.135061 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699808e7-d566-4f63-81c2-601932ff9fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:52.135344 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699808e7-d566-4f63-81c2-601932ff9fbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:52.136159 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699808e7-d566-4f63-81c2-601932ff9fbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:52.137874 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699808e7-d566-4f63-81c2-601932ff9fbc admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:52.155244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1305/2607] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:42:52.155590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1193f65b-61f5-4a4f-b2ce-813406c2fe8d admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:52.166720 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1193f65b-61f5-4a4f-b2ce-813406c2fe8d admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:52.174035 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969910f-dac4-4407-9035-92d288f049b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:52.174194 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969910f-dac4-4407-9035-92d288f049b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:52.174437 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969910f-dac4-4407-9035-92d288f049b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:52.174714 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969910f-dac4-4407-9035-92d288f049b9 admin admin] PATH_INFO: `/v3/projects/2645fbb40b7140a3ba86a6f8b814ca85/users/a6440a4d07334fb69bebbf38d9ba7602/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:52.191283 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6969910f-dac4-4407-9035-92d288f049b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2645fbb40b7140a3ba86a6f8b814ca85, user_id=a6440a4d07334fb69bebbf38d9ba7602, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:52.197030 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6969910f-dac4-4407-9035-92d288f049b9 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:52.215130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699808e7-d566-4f63-81c2-601932ff9fbc admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:52.217636 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-699808e7-d566-4f63-81c2-601932ff9fbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:52.240451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699808e7-d566-4f63-81c2-601932ff9fbc admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:52.246478 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2c195-717b-4235-991c-91a0c42616ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:52.248042 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2c195-717b-4235-991c-91a0c42616ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:52.248337 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2c195-717b-4235-991c-91a0c42616ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:52.248616 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2c195-717b-4235-991c-91a0c42616ca admin admin] PATH_INFO: `/v3/projects/a51d843f78a54e8b9dbba4ab9b28accf` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:52.249992 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f2c195-717b-4235-991c-91a0c42616ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=a51d843f78a54e8b9dbba4ab9b28accf)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:52.252163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f2c195-717b-4235-991c-91a0c42616ca admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:52.268190 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1303/2608] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:42:52 2022] PUT /identity/v3/projects/2645fbb40b7140a3ba86a6f8b814ca85/users/a6440a4d07334fb69bebbf38d9ba7602/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:52.281730 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602b41d-1e42-4bb6-9531-4fb39bfe72b2 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:52.282048 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602b41d-1e42-4bb6-9531-4fb39bfe72b2 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:52.282327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602b41d-1e42-4bb6-9531-4fb39bfe72b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:52.309579 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-9602b41d-1e42-4bb6-9531-4fb39bfe72b2 None None] MFA Rules not processed for user `a6440a4d07334fb69bebbf38d9ba7602`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:52.310792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-9602b41d-1e42-4bb6-9531-4fb39bfe72b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:52.379482 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-83f2c195-717b-4235-991c-91a0c42616ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a51d843f78a54e8b9dbba4ab9b28accf', 'request_id': 'req-83f2c195-717b-4235-991c-91a0c42616ca'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:52.380631 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1304/2609] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:42:52.385293 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602b41d-1e42-4bb6-9531-4fb39bfe72b2 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:52.397848 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:52.445903 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602b41d-1e42-4bb6-9531-4fb39bfe72b2 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:52.485977 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-83f2c195-717b-4235-991c-91a0c42616ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a51d843f78a54e8b9dbba4ab9b28accf', 'request_id': 'req-83f2c195-717b-4235-991c-91a0c42616ca'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:52.492832 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-83f2c195-717b-4235-991c-91a0c42616ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a51d843f78a54e8b9dbba4ab9b28accf', 'request_id': 'req-83f2c195-717b-4235-991c-91a0c42616ca'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:52.501824 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd742c9f-c501-45a7-a2b2-faf91791e8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:52.502554 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd742c9f-c501-45a7-a2b2-faf91791e8dc admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:52.502855 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd742c9f-c501-45a7-a2b2-faf91791e8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:52.503173 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd742c9f-c501-45a7-a2b2-faf91791e8dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:52.503903 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd742c9f-c501-45a7-a2b2-faf91791e8dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:52.504581 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-83f2c195-717b-4235-991c-91a0c42616ca admin admin] The token cache is being invalidate because project a51d843f78a54e8b9dbba4ab9b28accf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:42:52.504932 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-83f2c195-717b-4235-991c-91a0c42616ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:52.505748 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-83f2c195-717b-4235-991c-91a0c42616ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:52.505907 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd742c9f-c501-45a7-a2b2-faf91791e8dc admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:52.507455 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1306/2610] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:42:52 2022] DELETE /identity/v3/projects/a51d843f78a54e8b9dbba4ab9b28accf => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:52.512109 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f38da-67bf-4100-983b-368b392cbf01 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:52.512391 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f38da-67bf-4100-983b-368b392cbf01 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:52.512667 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f38da-67bf-4100-983b-368b392cbf01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:52.529626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-c46f38da-67bf-4100-983b-368b392cbf01 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:52.531256 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c46f38da-67bf-4100-983b-368b392cbf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:52.539279 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1307/2611] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:42:52.546803 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185709d-7ec5-486a-8d40-c7eca34d5051 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:52.547136 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185709d-7ec5-486a-8d40-c7eca34d5051 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:52.547455 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185709d-7ec5-486a-8d40-c7eca34d5051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:52.565814 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-1185709d-7ec5-486a-8d40-c7eca34d5051 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:52.567036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1185709d-7ec5-486a-8d40-c7eca34d5051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:52.574949 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1308/2612] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:42:52.579086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185709d-7ec5-486a-8d40-c7eca34d5051 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:52.585550 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1305/2613] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:42:52 2022] POST /identity/v3/projects => generated 387 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:42:52.589460 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185709d-7ec5-486a-8d40-c7eca34d5051 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:52.590663 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd742c9f-c501-45a7-a2b2-faf91791e8dc admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:52.592368 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-fd742c9f-c501-45a7-a2b2-faf91791e8dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:52.654999 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17dcda-e5d0-4d83-946d-fb1818ba532f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:52.655679 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17dcda-e5d0-4d83-946d-fb1818ba532f admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:52.655942 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17dcda-e5d0-4d83-946d-fb1818ba532f admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:52.656213 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17dcda-e5d0-4d83-946d-fb1818ba532f admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:52.657001 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17dcda-e5d0-4d83-946d-fb1818ba532f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:42:52.657600 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd742c9f-c501-45a7-a2b2-faf91791e8dc admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:52.657731 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17dcda-e5d0-4d83-946d-fb1818ba532f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:52.659800 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17dcda-e5d0-4d83-946d-fb1818ba532f admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:52.665935 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c6dc5e-33cf-4aa1-909c-292293927299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:52.665935 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6dc5e-33cf-4aa1-909c-292293927299 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:52.665935 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6dc5e-33cf-4aa1-909c-292293927299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:52.665935 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6dc5e-33cf-4aa1-909c-292293927299 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:52.666308 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c6dc5e-33cf-4aa1-909c-292293927299 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:52.666676 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c6dc5e-33cf-4aa1-909c-292293927299 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:52.673363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1309/2614] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:42:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:42:52.678104 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17dcda-e5d0-4d83-946d-fb1818ba532f admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:52.687827 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17dcda-e5d0-4d83-946d-fb1818ba532f admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:52.692612 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345fba6-2bbe-4384-b025-ee7368001740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:52.693238 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345fba6-2bbe-4384-b025-ee7368001740 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:52.693514 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345fba6-2bbe-4384-b025-ee7368001740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:52.693786 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345fba6-2bbe-4384-b025-ee7368001740 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:52.694462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345fba6-2bbe-4384-b025-ee7368001740 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:52.696551 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345fba6-2bbe-4384-b025-ee7368001740 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:52.790225 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1306/2615] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:42:52 2022] POST /identity/v3/users => generated 300 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:42:52.793812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c6dc5e-33cf-4aa1-909c-292293927299 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:52.803070 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c6dc5e-33cf-4aa1-909c-292293927299 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:52.807703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc242b-73d2-44dd-a9b1-dd92fc0277bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:52.808330 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc242b-73d2-44dd-a9b1-dd92fc0277bd admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:52.808611 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc242b-73d2-44dd-a9b1-dd92fc0277bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:52.808884 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc242b-73d2-44dd-a9b1-dd92fc0277bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:52.809601 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc242b-73d2-44dd-a9b1-dd92fc0277bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:52.811217 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc242b-73d2-44dd-a9b1-dd92fc0277bd admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:52.822705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1307/2616] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:42:52.826809 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1310/2617] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:42:52 2022] POST /identity/v3/projects => generated 367 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:42:52.829757 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc242b-73d2-44dd-a9b1-dd92fc0277bd admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:52.831468 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345fba6-2bbe-4384-b025-ee7368001740 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:52.840195 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc242b-73d2-44dd-a9b1-dd92fc0277bd admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:52.892997 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:52.895037 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed0cca-17c5-4c4d-b66f-794a723aee6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:52.901294 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345fba6-2bbe-4384-b025-ee7368001740 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:52.901462 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed0cca-17c5-4c4d-b66f-794a723aee6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:52.901462 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed0cca-17c5-4c4d-b66f-794a723aee6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:52.901462 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed0cca-17c5-4c4d-b66f-794a723aee6f admin admin] PATH_INFO: `/v3/projects/2dfc3cfcb43f4d88a7ec19738ca915f9/users/0fde91c5fd994c64a0ae037ee313ebc8/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:52.901772 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af110fb8-8ba4-4e54-bf95-2ee02bff6afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:52.902318 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af110fb8-8ba4-4e54-bf95-2ee02bff6afe admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:52.902665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af110fb8-8ba4-4e54-bf95-2ee02bff6afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:52.903037 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af110fb8-8ba4-4e54-bf95-2ee02bff6afe admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:52.903793 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af110fb8-8ba4-4e54-bf95-2ee02bff6afe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:52.905773 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af110fb8-8ba4-4e54-bf95-2ee02bff6afe admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:52.911843 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ed0cca-17c5-4c4d-b66f-794a723aee6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dfc3cfcb43f4d88a7ec19738ca915f9, user_id=0fde91c5fd994c64a0ae037ee313ebc8, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:52.914549 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ed0cca-17c5-4c4d-b66f-794a723aee6f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:53.035105 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1311/2618] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:42:52 2022] POST /identity/v3/users => generated 284 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:42:53.039762 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af110fb8-8ba4-4e54-bf95-2ee02bff6afe admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:53.046992 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1308/2619] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:42:52 2022] PUT /identity/v3/projects/2dfc3cfcb43f4d88a7ec19738ca915f9/users/0fde91c5fd994c64a0ae037ee313ebc8/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:42:53.049497 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af110fb8-8ba4-4e54-bf95-2ee02bff6afe admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:53.050726 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed0cca-17c5-4c4d-b66f-794a723aee6f admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:53.107048 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c255c-a4f8-4595-b07a-56c25397b4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:53.108511 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c255c-a4f8-4595-b07a-56c25397b4ce admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:53.108511 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c255c-a4f8-4595-b07a-56c25397b4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:53.108511 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c255c-a4f8-4595-b07a-56c25397b4ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:53.108982 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c255c-a4f8-4595-b07a-56c25397b4ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:53.110569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c255c-a4f8-4595-b07a-56c25397b4ce admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:53.115232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed0cca-17c5-4c4d-b66f-794a723aee6f admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:53.119904 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8c6d3-156b-4001-ad1f-6786c8d41f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:53.120418 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1312/2620] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:42:53 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:42:53.120612 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8c6d3-156b-4001-ad1f-6786c8d41f8f admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:53.120849 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8c6d3-156b-4001-ad1f-6786c8d41f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:53.121125 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8c6d3-156b-4001-ad1f-6786c8d41f8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:53.121849 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8c6d3-156b-4001-ad1f-6786c8d41f8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:53.123758 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8c6d3-156b-4001-ad1f-6786c8d41f8f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:53.124352 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c255c-a4f8-4595-b07a-56c25397b4ce admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:53.134503 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1309/2621] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:42:53 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:42:53.137064 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c255c-a4f8-4595-b07a-56c25397b4ce admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:53.139853 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8c6d3-156b-4001-ad1f-6786c8d41f8f admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:53.142016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf98b02-dcba-4b7d-89ac-d0a296bc139a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:53.142776 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf98b02-dcba-4b7d-89ac-d0a296bc139a admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:53.143094 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf98b02-dcba-4b7d-89ac-d0a296bc139a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:53.143368 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf98b02-dcba-4b7d-89ac-d0a296bc139a admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:53.144086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf98b02-dcba-4b7d-89ac-d0a296bc139a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:53.146530 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf98b02-dcba-4b7d-89ac-d0a296bc139a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:53.149984 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8c6d3-156b-4001-ad1f-6786c8d41f8f admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:53.154879 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f784d5b9-4aa6-4318-8786-b6ee13a2e608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:53.155930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784d5b9-4aa6-4318-8786-b6ee13a2e608 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:53.156209 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784d5b9-4aa6-4318-8786-b6ee13a2e608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:53.156482 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f784d5b9-4aa6-4318-8786-b6ee13a2e608 admin admin] PATH_INFO: `/v3/projects/2dfc3cfcb43f4d88a7ec19738ca915f9/users/0fde91c5fd994c64a0ae037ee313ebc8/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:53.156888 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1313/2622] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:42:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:42:53.158495 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784d5b9-4aa6-4318-8786-b6ee13a2e608 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dfc3cfcb43f4d88a7ec19738ca915f9, user_id=0fde91c5fd994c64a0ae037ee313ebc8, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:53.160581 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf98b02-dcba-4b7d-89ac-d0a296bc139a admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:53.165882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f784d5b9-4aa6-4318-8786-b6ee13a2e608 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:53.170436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf98b02-dcba-4b7d-89ac-d0a296bc139a admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:53.172034 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1310/2623] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:42:53 2022] PUT /identity/v3/projects/2dfc3cfcb43f4d88a7ec19738ca915f9/users/0fde91c5fd994c64a0ae037ee313ebc8/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:42:53.182142 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ecb03-3bf6-4134-9954-5e6584233426 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:53.182434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ecb03-3bf6-4134-9954-5e6584233426 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:53.182722 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ecb03-3bf6-4134-9954-5e6584233426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:53.210922 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-821ecb03-3bf6-4134-9954-5e6584233426 None None] MFA Rules not processed for user `0fde91c5fd994c64a0ae037ee313ebc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:53.217444 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-821ecb03-3bf6-4134-9954-5e6584233426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:53.233248 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4b4986-bd9c-48c2-a6be-5723f6a3d7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:53.239430 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b4986-bd9c-48c2-a6be-5723f6a3d7ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:53.239430 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b4986-bd9c-48c2-a6be-5723f6a3d7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:53.239430 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4b4986-bd9c-48c2-a6be-5723f6a3d7ba admin admin] PATH_INFO: `/v3/projects/4f0673ff5607409e93aa644e5a7b5802/users/b9f457e5f28f4e958d83bbc81c38c3a1/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:53.242894 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:53.250614 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b4986-bd9c-48c2-a6be-5723f6a3d7ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f0673ff5607409e93aa644e5a7b5802, user_id=b9f457e5f28f4e958d83bbc81c38c3a1, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:53.253393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4b4986-bd9c-48c2-a6be-5723f6a3d7ba admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:53.286761 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1311/2624] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:42:53.297284 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ecb03-3bf6-4134-9954-5e6584233426 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:53.298655 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-821ecb03-3bf6-4134-9954-5e6584233426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:53.351556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1314/2625] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:42:53 2022] PUT /identity/v3/projects/4f0673ff5607409e93aa644e5a7b5802/users/b9f457e5f28f4e958d83bbc81c38c3a1/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:42:53.361737 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ecb03-3bf6-4134-9954-5e6584233426 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:53.362988 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2d358-2bf4-437b-adcd-0adca38fdbd2 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:53.363312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2d358-2bf4-437b-adcd-0adca38fdbd2 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:53.363688 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2d358-2bf4-437b-adcd-0adca38fdbd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:53.390505 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-c4e2d358-2bf4-437b-adcd-0adca38fdbd2 None None] MFA Rules not processed for user `b9f457e5f28f4e958d83bbc81c38c3a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:53.391742 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c4e2d358-2bf4-437b-adcd-0adca38fdbd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:53.409503 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342d539-06ae-4c73-b0d6-b9bc4fc4d587 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:53.410154 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342d539-06ae-4c73-b0d6-b9bc4fc4d587 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:53.410437 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342d539-06ae-4c73-b0d6-b9bc4fc4d587 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:53.410715 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2342d539-06ae-4c73-b0d6-b9bc4fc4d587 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:53.429618 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:53.466573 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1315/2626] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:53 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:42:53.477834 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2d358-2bf4-437b-adcd-0adca38fdbd2 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:53.479242 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c4e2d358-2bf4-437b-adcd-0adca38fdbd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:53.482945 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342d539-06ae-4c73-b0d6-b9bc4fc4d587 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:53.484395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2342d539-06ae-4c73-b0d6-b9bc4fc4d587 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:53.491538 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2d358-2bf4-437b-adcd-0adca38fdbd2 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:53.495325 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad498c-ea55-42c9-bfec-817140d74877 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:53.495959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad498c-ea55-42c9-bfec-817140d74877 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:53.496239 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad498c-ea55-42c9-bfec-817140d74877 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:53.496565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad498c-ea55-42c9-bfec-817140d74877 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:53.497391 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1312/2627] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:42:53 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:42:53.505299 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad498c-ea55-42c9-bfec-817140d74877 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:53.506897 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad498c-ea55-42c9-bfec-817140d74877 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:53.515721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1316/2628] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:42:53 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:42:53.526918 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:54.636962 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342d539-06ae-4c73-b0d6-b9bc4fc4d587 service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:54.697742 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2342d539-06ae-4c73-b0d6-b9bc4fc4d587 service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:54.702285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6dfa3-386d-45f1-8c40-bca374022907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:54.703178 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6dfa3-386d-45f1-8c40-bca374022907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:54.703459 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6dfa3-386d-45f1-8c40-bca374022907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:54.703732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6dfa3-386d-45f1-8c40-bca374022907 admin admin] PATH_INFO: `/v3/users/a6440a4d07334fb69bebbf38d9ba7602` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:54.704966 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6dfa3-386d-45f1-8c40-bca374022907 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6440a4d07334fb69bebbf38d9ba7602)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:54.707016 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6dfa3-386d-45f1-8c40-bca374022907 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:54.729376 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:54.898176 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-83e6dfa3-386d-45f1-8c40-bca374022907 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6440a4d07334fb69bebbf38d9ba7602'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:54.965230 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-83e6dfa3-386d-45f1-8c40-bca374022907 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6440a4d07334fb69bebbf38d9ba7602'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:54.965801 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-83e6dfa3-386d-45f1-8c40-bca374022907 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6440a4d07334fb69bebbf38d9ba7602'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:54.972819 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-83e6dfa3-386d-45f1-8c40-bca374022907 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6440a4d07334fb69bebbf38d9ba7602'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:54.973465 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-83e6dfa3-386d-45f1-8c40-bca374022907 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6440a4d07334fb69bebbf38d9ba7602'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:54.993329 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1313/2629] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:42:54 2022] DELETE /identity/v3/users/a6440a4d07334fb69bebbf38d9ba7602 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:54.998022 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc124ea-ea95-4466-8d9c-9ec03d6d0ca2 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:55.001839 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc124ea-ea95-4466-8d9c-9ec03d6d0ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:55.001839 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc124ea-ea95-4466-8d9c-9ec03d6d0ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:55.016952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-ccc124ea-ea95-4466-8d9c-9ec03d6d0ca2 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:55.018143 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ccc124ea-ea95-4466-8d9c-9ec03d6d0ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:55.075664 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1317/2630] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:42:55.085677 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6dfa3-386d-45f1-8c40-bca374022907 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:55.087117 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-83e6dfa3-386d-45f1-8c40-bca374022907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:55.142965 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6dfa3-386d-45f1-8c40-bca374022907 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:55.146687 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dd734-9880-47b3-b916-ab944c99b5f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:55.147351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dd734-9880-47b3-b916-ab944c99b5f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:55.153109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dd734-9880-47b3-b916-ab944c99b5f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:55.153109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dd734-9880-47b3-b916-ab944c99b5f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:55.157022 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69dd734-9880-47b3-b916-ab944c99b5f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:55.158470 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69dd734-9880-47b3-b916-ab944c99b5f5 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:55.168088 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1314/2631] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:42:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:55.633680 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc124ea-ea95-4466-8d9c-9ec03d6d0ca2 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:55.635208 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ccc124ea-ea95-4466-8d9c-9ec03d6d0ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:55.655780 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc124ea-ea95-4466-8d9c-9ec03d6d0ca2 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:55.665410 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7faa2c-3807-4971-90b6-9f4af9f8c074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:55.667424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7faa2c-3807-4971-90b6-9f4af9f8c074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:55.668773 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7faa2c-3807-4971-90b6-9f4af9f8c074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:55.669295 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7faa2c-3807-4971-90b6-9f4af9f8c074 admin admin] PATH_INFO: `/v3/projects/2645fbb40b7140a3ba86a6f8b814ca85` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:55.671374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7faa2c-3807-4971-90b6-9f4af9f8c074 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2645fbb40b7140a3ba86a6f8b814ca85)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:55.676023 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7faa2c-3807-4971-90b6-9f4af9f8c074 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:55.702594 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:55.806683 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7c7faa2c-3807-4971-90b6-9f4af9f8c074 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2645fbb40b7140a3ba86a6f8b814ca85', 'request_id': 'req-7c7faa2c-3807-4971-90b6-9f4af9f8c074'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:55.944085 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7c7faa2c-3807-4971-90b6-9f4af9f8c074 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2645fbb40b7140a3ba86a6f8b814ca85', 'request_id': 'req-7c7faa2c-3807-4971-90b6-9f4af9f8c074'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:55.950092 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7c7faa2c-3807-4971-90b6-9f4af9f8c074 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2645fbb40b7140a3ba86a6f8b814ca85', 'request_id': 'req-7c7faa2c-3807-4971-90b6-9f4af9f8c074'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:55.961468 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7c7faa2c-3807-4971-90b6-9f4af9f8c074 admin admin] The token cache is being invalidate because project 2645fbb40b7140a3ba86a6f8b814ca85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:42:55.961782 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7c7faa2c-3807-4971-90b6-9f4af9f8c074 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:55.962423 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-7c7faa2c-3807-4971-90b6-9f4af9f8c074 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:55.964021 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1318/2632] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:42:55 2022] DELETE /identity/v3/projects/2645fbb40b7140a3ba86a6f8b814ca85 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:42:55.969601 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dd734-9880-47b3-b916-ab944c99b5f5 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:55.983626 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dd734-9880-47b3-b916-ab944c99b5f5 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:56.047142 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4ef8d-ce52-461a-afe7-f7dc16d631d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:56.048028 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4ef8d-ce52-461a-afe7-f7dc16d631d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:56.048309 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4ef8d-ce52-461a-afe7-f7dc16d631d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:56.048584 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4ef8d-ce52-461a-afe7-f7dc16d631d4 admin admin] PATH_INFO: `/v3/users/0fde91c5fd994c64a0ae037ee313ebc8` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:56.049776 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4ef8d-ce52-461a-afe7-f7dc16d631d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fde91c5fd994c64a0ae037ee313ebc8)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:56.052451 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4ef8d-ce52-461a-afe7-f7dc16d631d4 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:56.291506 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:56.293017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-06e4ef8d-ce52-461a-afe7-f7dc16d631d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fde91c5fd994c64a0ae037ee313ebc8'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:56.360079 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-06e4ef8d-ce52-461a-afe7-f7dc16d631d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fde91c5fd994c64a0ae037ee313ebc8'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:56.360605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-06e4ef8d-ce52-461a-afe7-f7dc16d631d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fde91c5fd994c64a0ae037ee313ebc8'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:56.368764 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-06e4ef8d-ce52-461a-afe7-f7dc16d631d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fde91c5fd994c64a0ae037ee313ebc8'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:56.375240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-06e4ef8d-ce52-461a-afe7-f7dc16d631d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fde91c5fd994c64a0ae037ee313ebc8'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:56.386321 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1315/2633] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:42:55 2022] DELETE /identity/v3/users/0fde91c5fd994c64a0ae037ee313ebc8 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:56.763316 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7faa2c-3807-4971-90b6-9f4af9f8c074 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:56.764866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-7c7faa2c-3807-4971-90b6-9f4af9f8c074 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:56.781952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7faa2c-3807-4971-90b6-9f4af9f8c074 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:56.851303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43be94d6-e83b-48cf-83b8-45eb2aae19c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:56.852346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be94d6-e83b-48cf-83b8-45eb2aae19c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:56.852718 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be94d6-e83b-48cf-83b8-45eb2aae19c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:56.853088 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43be94d6-e83b-48cf-83b8-45eb2aae19c2 admin admin] PATH_INFO: `/v3/projects/2dfc3cfcb43f4d88a7ec19738ca915f9` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:56.854365 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43be94d6-e83b-48cf-83b8-45eb2aae19c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dfc3cfcb43f4d88a7ec19738ca915f9)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:56.856305 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43be94d6-e83b-48cf-83b8-45eb2aae19c2 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:56.996592 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-43be94d6-e83b-48cf-83b8-45eb2aae19c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dfc3cfcb43f4d88a7ec19738ca915f9', 'request_id': 'req-43be94d6-e83b-48cf-83b8-45eb2aae19c2'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:57.085345 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-43be94d6-e83b-48cf-83b8-45eb2aae19c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dfc3cfcb43f4d88a7ec19738ca915f9', 'request_id': 'req-43be94d6-e83b-48cf-83b8-45eb2aae19c2'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:57.091421 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-43be94d6-e83b-48cf-83b8-45eb2aae19c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dfc3cfcb43f4d88a7ec19738ca915f9', 'request_id': 'req-43be94d6-e83b-48cf-83b8-45eb2aae19c2'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:57.103060 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-43be94d6-e83b-48cf-83b8-45eb2aae19c2 admin admin] The token cache is being invalidate because project 2dfc3cfcb43f4d88a7ec19738ca915f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:42:57.106058 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-43be94d6-e83b-48cf-83b8-45eb2aae19c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:57.106058 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-43be94d6-e83b-48cf-83b8-45eb2aae19c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:57.106058 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1319/2634] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:42:56 2022] DELETE /identity/v3/projects/2dfc3cfcb43f4d88a7ec19738ca915f9 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:57.110588 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d3efc-b684-4442-be39-9ebfc495502e None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:57.111099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d3efc-b684-4442-be39-9ebfc495502e None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:57.111382 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d3efc-b684-4442-be39-9ebfc495502e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:57.130047 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-229d3efc-b684-4442-be39-9ebfc495502e None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:57.131268 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-229d3efc-b684-4442-be39-9ebfc495502e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:57.189709 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1316/2635] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:42:57.198619 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc272b6-946b-4b53-9bdd-d19dbe9d99ee None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:57.198961 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc272b6-946b-4b53-9bdd-d19dbe9d99ee None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:57.199296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc272b6-946b-4b53-9bdd-d19dbe9d99ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:57.217567 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-ddc272b6-946b-4b53-9bdd-d19dbe9d99ee None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:57.220136 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ddc272b6-946b-4b53-9bdd-d19dbe9d99ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:57.227303 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1320/2636] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:57.231442 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d3efc-b684-4442-be39-9ebfc495502e None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:57.241517 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d3efc-b684-4442-be39-9ebfc495502e None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:57.246054 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22195d9f-0647-47c2-9c77-9c342e8f1ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:57.246697 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22195d9f-0647-47c2-9c77-9c342e8f1ab4 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:57.247005 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22195d9f-0647-47c2-9c77-9c342e8f1ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:57.247279 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22195d9f-0647-47c2-9c77-9c342e8f1ab4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:57.248002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22195d9f-0647-47c2-9c77-9c342e8f1ab4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:42:57.248311 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22195d9f-0647-47c2-9c77-9c342e8f1ab4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:57.249897 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22195d9f-0647-47c2-9c77-9c342e8f1ab4 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:57.261054 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1317/2637] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:42:57.265307 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc272b6-946b-4b53-9bdd-d19dbe9d99ee None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:57.275687 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc272b6-946b-4b53-9bdd-d19dbe9d99ee None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:57.280589 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7994a-ab1e-44e8-bd79-c4dd19e50641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:57.281237 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7994a-ab1e-44e8-bd79-c4dd19e50641 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:57.281518 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7994a-ab1e-44e8-bd79-c4dd19e50641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:57.281794 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7994a-ab1e-44e8-bd79-c4dd19e50641 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:57.282525 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7994a-ab1e-44e8-bd79-c4dd19e50641 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:57.284532 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7994a-ab1e-44e8-bd79-c4dd19e50641 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:57.414591 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1321/2638] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:42:57 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:42:57.418653 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22195d9f-0647-47c2-9c77-9c342e8f1ab4 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:57.428739 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22195d9f-0647-47c2-9c77-9c342e8f1ab4 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:57.486398 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff745a1b-551d-42e1-a0f3-b82f156c8017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:57.487116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff745a1b-551d-42e1-a0f3-b82f156c8017 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:57.487406 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff745a1b-551d-42e1-a0f3-b82f156c8017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:57.487687 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff745a1b-551d-42e1-a0f3-b82f156c8017 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:57.488374 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff745a1b-551d-42e1-a0f3-b82f156c8017 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:57.490307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff745a1b-551d-42e1-a0f3-b82f156c8017 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:57.606694 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1318/2639] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:42:57 2022] POST /identity/v3/users => generated 287 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:42:57.612717 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7994a-ab1e-44e8-bd79-c4dd19e50641 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:57.619638 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:57.623374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7994a-ab1e-44e8-bd79-c4dd19e50641 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:57.628181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a2537-5d01-4f4f-ad3b-46638cca0a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:57.628619 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a2537-5d01-4f4f-ad3b-46638cca0a60 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:57.628903 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a2537-5d01-4f4f-ad3b-46638cca0a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:57.629181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a2537-5d01-4f4f-ad3b-46638cca0a60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:57.630428 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a2537-5d01-4f4f-ad3b-46638cca0a60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:57.632205 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a2537-5d01-4f4f-ad3b-46638cca0a60 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:57.644990 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1322/2640] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:42:57.649720 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff745a1b-551d-42e1-a0f3-b82f156c8017 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:57.660424 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff745a1b-551d-42e1-a0f3-b82f156c8017 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:57.667842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede66081-56d8-4713-88cd-a921c88cfabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:57.667842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede66081-56d8-4713-88cd-a921c88cfabe admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:57.667842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede66081-56d8-4713-88cd-a921c88cfabe admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:57.667842 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede66081-56d8-4713-88cd-a921c88cfabe admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:57.668430 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede66081-56d8-4713-88cd-a921c88cfabe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:57.669015 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede66081-56d8-4713-88cd-a921c88cfabe admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:57.682754 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1319/2641] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:42:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:42:57.683996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a2537-5d01-4f4f-ad3b-46638cca0a60 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:57.694454 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a2537-5d01-4f4f-ad3b-46638cca0a60 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:57.699182 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44716e-3db1-4348-98d2-c2699f04e920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:57.700065 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44716e-3db1-4348-98d2-c2699f04e920 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:57.700351 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44716e-3db1-4348-98d2-c2699f04e920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:57.700625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44716e-3db1-4348-98d2-c2699f04e920 admin admin] PATH_INFO: `/v3/projects/4f284e71009d4f17a4db35b768022d0d/users/ab38ffdede834227bb157d1f2a217a39/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:57.716012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44716e-3db1-4348-98d2-c2699f04e920 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f284e71009d4f17a4db35b768022d0d, user_id=ab38ffdede834227bb157d1f2a217a39, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:57.718656 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44716e-3db1-4348-98d2-c2699f04e920 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:57.789296 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1323/2642] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:42:57 2022] PUT /identity/v3/projects/4f284e71009d4f17a4db35b768022d0d/users/ab38ffdede834227bb157d1f2a217a39/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:42:57.799611 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40dccc-59aa-4fc8-a720-a4dff9b39e55 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:57.799903 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40dccc-59aa-4fc8-a720-a4dff9b39e55 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:57.800180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d40dccc-59aa-4fc8-a720-a4dff9b39e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:57.826646 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-9d40dccc-59aa-4fc8-a720-a4dff9b39e55 None None] MFA Rules not processed for user `ab38ffdede834227bb157d1f2a217a39`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:57.827941 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-9d40dccc-59aa-4fc8-a720-a4dff9b39e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:57.845054 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:57.890344 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1320/2643] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:57.902451 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44716e-3db1-4348-98d2-c2699f04e920 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:57.904142 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-aa44716e-3db1-4348-98d2-c2699f04e920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:57.963468 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44716e-3db1-4348-98d2-c2699f04e920 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:57.967112 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7c10c-96bd-4a42-b03b-6b7301aec8c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:57.967753 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7c10c-96bd-4a42-b03b-6b7301aec8c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:57.968036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7c10c-96bd-4a42-b03b-6b7301aec8c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:57.968312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7c10c-96bd-4a42-b03b-6b7301aec8c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:57.977432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c7c10c-96bd-4a42-b03b-6b7301aec8c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:57.978891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c7c10c-96bd-4a42-b03b-6b7301aec8c6 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:57.988248 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1324/2644] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:42:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:58.009342 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40dccc-59aa-4fc8-a720-a4dff9b39e55 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:58.072323 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d40dccc-59aa-4fc8-a720-a4dff9b39e55 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:58.076906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e56b8-4aa8-498a-a01b-747c54a89783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:58.077774 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e56b8-4aa8-498a-a01b-747c54a89783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:58.078053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e56b8-4aa8-498a-a01b-747c54a89783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:58.078978 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e56b8-4aa8-498a-a01b-747c54a89783 admin admin] PATH_INFO: `/v3/users/ab38ffdede834227bb157d1f2a217a39` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:58.079650 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e56b8-4aa8-498a-a01b-747c54a89783 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab38ffdede834227bb157d1f2a217a39)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:58.081493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e56b8-4aa8-498a-a01b-747c54a89783 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:58.259904 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7f9e56b8-4aa8-498a-a01b-747c54a89783 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab38ffdede834227bb157d1f2a217a39'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:58.325776 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7f9e56b8-4aa8-498a-a01b-747c54a89783 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab38ffdede834227bb157d1f2a217a39'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:58.326428 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7f9e56b8-4aa8-498a-a01b-747c54a89783 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab38ffdede834227bb157d1f2a217a39'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:58.333629 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7f9e56b8-4aa8-498a-a01b-747c54a89783 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab38ffdede834227bb157d1f2a217a39'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:58.334271 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-7f9e56b8-4aa8-498a-a01b-747c54a89783 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab38ffdede834227bb157d1f2a217a39'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:58.351646 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1321/2645] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:42:58 2022] DELETE /identity/v3/users/ab38ffdede834227bb157d1f2a217a39 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:58.356166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374506bb-381f-4dff-afd0-f1ad8232eaf9 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:58.356454 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374506bb-381f-4dff-afd0-f1ad8232eaf9 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:58.356728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374506bb-381f-4dff-afd0-f1ad8232eaf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:58.374611 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-374506bb-381f-4dff-afd0-f1ad8232eaf9 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:58.375980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-374506bb-381f-4dff-afd0-f1ad8232eaf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:58.435520 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1325/2646] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:42:58.445053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e56b8-4aa8-498a-a01b-747c54a89783 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:58.446505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-7f9e56b8-4aa8-498a-a01b-747c54a89783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:58.487770 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:58.502954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e56b8-4aa8-498a-a01b-747c54a89783 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:58.507096 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048537e2-63d1-437b-8b6c-b68833b1175f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:58.507729 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048537e2-63d1-437b-8b6c-b68833b1175f service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:58.508005 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048537e2-63d1-437b-8b6c-b68833b1175f service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:58.508294 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048537e2-63d1-437b-8b6c-b68833b1175f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:58.517060 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048537e2-63d1-437b-8b6c-b68833b1175f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:58.518430 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048537e2-63d1-437b-8b6c-b68833b1175f service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:58.528289 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1322/2647] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:42:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:42:59.008614 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374506bb-381f-4dff-afd0-f1ad8232eaf9 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:59.010085 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-374506bb-381f-4dff-afd0-f1ad8232eaf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:59.029759 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374506bb-381f-4dff-afd0-f1ad8232eaf9 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:59.033010 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81694ca-6a10-42ca-a25b-19f1436d2e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:59.033878 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81694ca-6a10-42ca-a25b-19f1436d2e98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:59.034158 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81694ca-6a10-42ca-a25b-19f1436d2e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:59.034445 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81694ca-6a10-42ca-a25b-19f1436d2e98 admin admin] PATH_INFO: `/v3/projects/4f284e71009d4f17a4db35b768022d0d` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:59.035688 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81694ca-6a10-42ca-a25b-19f1436d2e98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f284e71009d4f17a4db35b768022d0d)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:59.037666 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81694ca-6a10-42ca-a25b-19f1436d2e98 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:59.164704 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a81694ca-6a10-42ca-a25b-19f1436d2e98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f284e71009d4f17a4db35b768022d0d', 'request_id': 'req-a81694ca-6a10-42ca-a25b-19f1436d2e98'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:59.292309 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a81694ca-6a10-42ca-a25b-19f1436d2e98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f284e71009d4f17a4db35b768022d0d', 'request_id': 'req-a81694ca-6a10-42ca-a25b-19f1436d2e98'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:59.298791 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a81694ca-6a10-42ca-a25b-19f1436d2e98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f284e71009d4f17a4db35b768022d0d', 'request_id': 'req-a81694ca-6a10-42ca-a25b-19f1436d2e98'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:59.300821 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:42:59.311295 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a81694ca-6a10-42ca-a25b-19f1436d2e98 admin admin] The token cache is being invalidate because project 4f284e71009d4f17a4db35b768022d0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:42:59.311655 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a81694ca-6a10-42ca-a25b-19f1436d2e98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:59.312370 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-a81694ca-6a10-42ca-a25b-19f1436d2e98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:42:59.314053 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1326/2648] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:42:59 2022] DELETE /identity/v3/projects/4f284e71009d4f17a4db35b768022d0d => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:42:59.318953 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457033f-2563-4c05-a8b8-06740181392c None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:59.319426 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457033f-2563-4c05-a8b8-06740181392c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:59.319600 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457033f-2563-4c05-a8b8-06740181392c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:59.337705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-4457033f-2563-4c05-a8b8-06740181392c None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:59.338942 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-4457033f-2563-4c05-a8b8-06740181392c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:59.400054 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1323/2649] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:59.407928 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f18c3c-3803-4849-a6f0-ee0ffdd5e13f None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:59.408219 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f18c3c-3803-4849-a6f0-ee0ffdd5e13f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:59.408498 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f18c3c-3803-4849-a6f0-ee0ffdd5e13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:59.428074 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-30f18c3c-3803-4849-a6f0-ee0ffdd5e13f None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:42:59.432126 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-30f18c3c-3803-4849-a6f0-ee0ffdd5e13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:42:59.437206 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1327/2650] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:42:59.441669 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457033f-2563-4c05-a8b8-06740181392c None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:59.451639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457033f-2563-4c05-a8b8-06740181392c None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:59.456866 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91676d90-aadf-4ccd-9117-91a6766392c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:59.457597 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91676d90-aadf-4ccd-9117-91a6766392c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:59.458024 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91676d90-aadf-4ccd-9117-91a6766392c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:59.458476 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91676d90-aadf-4ccd-9117-91a6766392c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:59.459236 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91676d90-aadf-4ccd-9117-91a6766392c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:42:59.459546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91676d90-aadf-4ccd-9117-91a6766392c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:59.461188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91676d90-aadf-4ccd-9117-91a6766392c2 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:59.474570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1324/2651] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:42:59.478752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f18c3c-3803-4849-a6f0-ee0ffdd5e13f None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:59.487995 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f18c3c-3803-4849-a6f0-ee0ffdd5e13f None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:59.493438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f87b1-62c6-424a-8a6b-61cd7c5e2cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:59.494125 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f87b1-62c6-424a-8a6b-61cd7c5e2cb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:59.494407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f87b1-62c6-424a-8a6b-61cd7c5e2cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:59.494685 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f87b1-62c6-424a-8a6b-61cd7c5e2cb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:59.495556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f87b1-62c6-424a-8a6b-61cd7c5e2cb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:59.497665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f87b1-62c6-424a-8a6b-61cd7c5e2cb8 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:59.624915 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1328/2652] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:42:59 2022] POST /identity/v3/projects => generated 381 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:42:59.629190 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91676d90-aadf-4ccd-9117-91a6766392c2 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:59.639420 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91676d90-aadf-4ccd-9117-91a6766392c2 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:59.699025 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494a365-2ea0-4771-bf2e-9029bf21c507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:59.699704 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494a365-2ea0-4771-bf2e-9029bf21c507 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:59.699992 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494a365-2ea0-4771-bf2e-9029bf21c507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:59.700493 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494a365-2ea0-4771-bf2e-9029bf21c507 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:59.701803 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6494a365-2ea0-4771-bf2e-9029bf21c507 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:59.704441 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6494a365-2ea0-4771-bf2e-9029bf21c507 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:59.813053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1325/2653] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:42:59 2022] POST /identity/v3/users => generated 291 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:42:59.816486 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f87b1-62c6-424a-8a6b-61cd7c5e2cb8 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:59.827325 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f87b1-62c6-424a-8a6b-61cd7c5e2cb8 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:59.831945 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5709f-71af-494b-9220-a5d28e3457ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:59.832583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5709f-71af-494b-9220-a5d28e3457ae admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:59.832927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5709f-71af-494b-9220-a5d28e3457ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:59.833457 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5709f-71af-494b-9220-a5d28e3457ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:59.834205 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e5709f-71af-494b-9220-a5d28e3457ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:59.835859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e5709f-71af-494b-9220-a5d28e3457ae admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:59.846685 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1329/2654] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:42:59.852964 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494a365-2ea0-4771-bf2e-9029bf21c507 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:59.863244 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494a365-2ea0-4771-bf2e-9029bf21c507 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:59.868644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55972bab-92f3-4044-a52a-d981da786ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:59.869305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55972bab-92f3-4044-a52a-d981da786ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:59.869592 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55972bab-92f3-4044-a52a-d981da786ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:59.869870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55972bab-92f3-4044-a52a-d981da786ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:59.870603 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55972bab-92f3-4044-a52a-d981da786ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:59.872287 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55972bab-92f3-4044-a52a-d981da786ba4 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:42:59.882222 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1326/2655] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:42:59.886137 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5709f-71af-494b-9220-a5d28e3457ae admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:42:59.896133 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5709f-71af-494b-9220-a5d28e3457ae admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:42:59.900656 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc204c47-7576-4ef6-9d69-8ee703fd8ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:42:59.901519 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc204c47-7576-4ef6-9d69-8ee703fd8ce1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:42:59.901827 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc204c47-7576-4ef6-9d69-8ee703fd8ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:42:59.902100 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc204c47-7576-4ef6-9d69-8ee703fd8ce1 admin admin] PATH_INFO: `/v3/projects/245626825e3b480485223cd0e3d54580/users/1e33fbb258684a00a7a8d948d92a8c01/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:42:59.916658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc204c47-7576-4ef6-9d69-8ee703fd8ce1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=245626825e3b480485223cd0e3d54580, user_id=1e33fbb258684a00a7a8d948d92a8c01, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:42:59.919314 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc204c47-7576-4ef6-9d69-8ee703fd8ce1 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:00.031721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1330/2656] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:42:59 2022] PUT /identity/v3/projects/245626825e3b480485223cd0e3d54580/users/1e33fbb258684a00a7a8d948d92a8c01/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:43:00.037958 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe7a03-059b-4899-bc0b-21a09c456e71 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:00.038259 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe7a03-059b-4899-bc0b-21a09c456e71 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:00.038542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe7a03-059b-4899-bc0b-21a09c456e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:00.060170 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-51fe7a03-059b-4899-bc0b-21a09c456e71 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:00.061423 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-51fe7a03-059b-4899-bc0b-21a09c456e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:00.104023 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:43:00.120766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1327/2657] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:43:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:00.131284 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc204c47-7576-4ef6-9d69-8ee703fd8ce1 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:00.133053 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-fc204c47-7576-4ef6-9d69-8ee703fd8ce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:00.189041 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc204c47-7576-4ef6-9d69-8ee703fd8ce1 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:00.193147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc21ed-5bfd-4291-9bb6-1e1cf29f9f51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:00.193778 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc21ed-5bfd-4291-9bb6-1e1cf29f9f51 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:00.194057 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc21ed-5bfd-4291-9bb6-1e1cf29f9f51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:00.194329 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc21ed-5bfd-4291-9bb6-1e1cf29f9f51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:00.203154 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cc21ed-5bfd-4291-9bb6-1e1cf29f9f51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:00.204579 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cc21ed-5bfd-4291-9bb6-1e1cf29f9f51 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:00.214980 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1331/2658] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:43:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:43:01.709841 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04a574-039d-451b-9f9e-5f05e6329d2e None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:01.710421 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04a574-039d-451b-9f9e-5f05e6329d2e None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:01.710595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04a574-039d-451b-9f9e-5f05e6329d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:01.738316 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-4d04a574-039d-451b-9f9e-5f05e6329d2e None None] MFA Rules not processed for user `1e33fbb258684a00a7a8d948d92a8c01`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:01.740092 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-4d04a574-039d-451b-9f9e-5f05e6329d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:01.803759 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1328/2659] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:43:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:01.814306 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc21ed-5bfd-4291-9bb6-1e1cf29f9f51 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:01.815772 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-09cc21ed-5bfd-4291-9bb6-1e1cf29f9f51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:01.871407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc21ed-5bfd-4291-9bb6-1e1cf29f9f51 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:01.875138 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c967b-8d91-47fb-8eb3-d9f9f34eec34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:01.875801 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c967b-8d91-47fb-8eb3-d9f9f34eec34 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:01.876085 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c967b-8d91-47fb-8eb3-d9f9f34eec34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:01.876360 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c967b-8d91-47fb-8eb3-d9f9f34eec34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:01.885702 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c967b-8d91-47fb-8eb3-d9f9f34eec34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:01.887175 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c967b-8d91-47fb-8eb3-d9f9f34eec34 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:01.896136 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1332/2660] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:43:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:43:03.426702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04a574-039d-451b-9f9e-5f05e6329d2e None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:03.428147 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-4d04a574-039d-451b-9f9e-5f05e6329d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:03.441664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04a574-039d-451b-9f9e-5f05e6329d2e None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:03.446217 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34795da3-62a8-4192-9731-90cd917c94f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:03.447110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34795da3-62a8-4192-9731-90cd917c94f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:03.449152 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34795da3-62a8-4192-9731-90cd917c94f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:03.449152 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34795da3-62a8-4192-9731-90cd917c94f1 admin admin] PATH_INFO: `/v3/users/b9f457e5f28f4e958d83bbc81c38c3a1` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:03.449152 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34795da3-62a8-4192-9731-90cd917c94f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9f457e5f28f4e958d83bbc81c38c3a1)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:03.450775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34795da3-62a8-4192-9731-90cd917c94f1 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:03.652287 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e799e-3b5c-487d-b86b-753b87841948 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:03.652287 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e799e-3b5c-487d-b86b-753b87841948 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:03.652287 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e799e-3b5c-487d-b86b-753b87841948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:03.654187 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-d58e799e-3b5c-487d-b86b-753b87841948 None None] MFA Rules not processed for user `1e33fbb258684a00a7a8d948d92a8c01`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:03.655441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d58e799e-3b5c-487d-b86b-753b87841948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:03.662369 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1333/2661] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:43:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:43:03.695167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-34795da3-62a8-4192-9731-90cd917c94f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9f457e5f28f4e958d83bbc81c38c3a1'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:03.835153 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-34795da3-62a8-4192-9731-90cd917c94f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9f457e5f28f4e958d83bbc81c38c3a1'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:03.835854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-34795da3-62a8-4192-9731-90cd917c94f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9f457e5f28f4e958d83bbc81c38c3a1'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:03.844243 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-34795da3-62a8-4192-9731-90cd917c94f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9f457e5f28f4e958d83bbc81c38c3a1'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:03.844996 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-34795da3-62a8-4192-9731-90cd917c94f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9f457e5f28f4e958d83bbc81c38c3a1'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:03.862932 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1329/2662] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:43:03 2022] DELETE /identity/v3/users/b9f457e5f28f4e958d83bbc81c38c3a1 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:43:03.867176 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fba3db-7984-43b6-8199-7c367306cd53 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:03.867462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fba3db-7984-43b6-8199-7c367306cd53 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:03.867765 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fba3db-7984-43b6-8199-7c367306cd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:03.886209 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-d9fba3db-7984-43b6-8199-7c367306cd53 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:03.887561 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d9fba3db-7984-43b6-8199-7c367306cd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:03.914308 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:43:03.948374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1334/2663] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:43:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:03.959584 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34795da3-62a8-4192-9731-90cd917c94f1 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:03.960960 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-34795da3-62a8-4192-9731-90cd917c94f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:03.970391 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fba3db-7984-43b6-8199-7c367306cd53 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:03.971842 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d9fba3db-7984-43b6-8199-7c367306cd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:04.017542 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:43:04.021397 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34795da3-62a8-4192-9731-90cd917c94f1 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:04.025023 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e5ece-3d03-412f-83c5-fa994ff1041a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:04.025667 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e5ece-3d03-412f-83c5-fa994ff1041a service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:04.025951 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e5ece-3d03-412f-83c5-fa994ff1041a service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:04.026224 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e5ece-3d03-412f-83c5-fa994ff1041a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:04.030240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fba3db-7984-43b6-8199-7c367306cd53 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:04.033814 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18ba07-bc15-4469-9f5c-7f46ed89f094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:04.034489 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18ba07-bc15-4469-9f5c-7f46ed89f094 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:04.034774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18ba07-bc15-4469-9f5c-7f46ed89f094 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:04.035082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18ba07-bc15-4469-9f5c-7f46ed89f094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:04.035318 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e5ece-3d03-412f-83c5-fa994ff1041a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:04.036815 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e5ece-3d03-412f-83c5-fa994ff1041a service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:04.036940 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6d18ba07-bc15-4469-9f5c-7f46ed89f094 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:04.046472 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18ba07-bc15-4469-9f5c-7f46ed89f094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:04.047110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1330/2664] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:43:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:43:04.047945 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18ba07-bc15-4469-9f5c-7f46ed89f094 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:04.057380 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1335/2665] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:43:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:43:04.470738 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e5ece-3d03-412f-83c5-fa994ff1041a service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:04.473074 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1b6e5ece-3d03-412f-83c5-fa994ff1041a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:04.488099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e5ece-3d03-412f-83c5-fa994ff1041a service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:04.505996 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6553646a-617f-4c83-98b8-fce28d6e2190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:04.506968 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553646a-617f-4c83-98b8-fce28d6e2190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:04.507251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553646a-617f-4c83-98b8-fce28d6e2190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:04.507664 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553646a-617f-4c83-98b8-fce28d6e2190 admin admin] PATH_INFO: `/v3/projects/4f0673ff5607409e93aa644e5a7b5802` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:04.509745 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6553646a-617f-4c83-98b8-fce28d6e2190 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f0673ff5607409e93aa644e5a7b5802)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:04.519597 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6553646a-617f-4c83-98b8-fce28d6e2190 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:04.640129 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6553646a-617f-4c83-98b8-fce28d6e2190 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f0673ff5607409e93aa644e5a7b5802', 'request_id': 'req-6553646a-617f-4c83-98b8-fce28d6e2190'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:04.736871 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6553646a-617f-4c83-98b8-fce28d6e2190 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f0673ff5607409e93aa644e5a7b5802', 'request_id': 'req-6553646a-617f-4c83-98b8-fce28d6e2190'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:04.743931 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6553646a-617f-4c83-98b8-fce28d6e2190 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f0673ff5607409e93aa644e5a7b5802', 'request_id': 'req-6553646a-617f-4c83-98b8-fce28d6e2190'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:04.754509 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6553646a-617f-4c83-98b8-fce28d6e2190 admin admin] The token cache is being invalidate because project 4f0673ff5607409e93aa644e5a7b5802 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:43:04.754797 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6553646a-617f-4c83-98b8-fce28d6e2190 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:04.755446 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-6553646a-617f-4c83-98b8-fce28d6e2190 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:04.756976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1331/2666] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:43:04 2022] DELETE /identity/v3/projects/4f0673ff5607409e93aa644e5a7b5802 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:43:04.763207 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710cf13-243b-4d2b-8acb-6b66b1b425c1 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:04.763711 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710cf13-243b-4d2b-8acb-6b66b1b425c1 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:04.763866 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710cf13-243b-4d2b-8acb-6b66b1b425c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:04.782068 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-6710cf13-243b-4d2b-8acb-6b66b1b425c1 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:04.783436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6710cf13-243b-4d2b-8acb-6b66b1b425c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:04.842075 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1336/2667] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:04.850661 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd061627-386f-4ceb-90d5-8f9e2be70b88 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:04.850983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd061627-386f-4ceb-90d5-8f9e2be70b88 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:04.851263 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd061627-386f-4ceb-90d5-8f9e2be70b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:04.869419 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-cd061627-386f-4ceb-90d5-8f9e2be70b88 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:04.870629 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-cd061627-386f-4ceb-90d5-8f9e2be70b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:04.878644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1332/2668] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:04.883283 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710cf13-243b-4d2b-8acb-6b66b1b425c1 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:04.892823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710cf13-243b-4d2b-8acb-6b66b1b425c1 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:04.897886 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f98a3-2562-47d2-b345-09322b8b9792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:04.898535 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f98a3-2562-47d2-b345-09322b8b9792 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:04.898817 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f98a3-2562-47d2-b345-09322b8b9792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:04.899124 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f98a3-2562-47d2-b345-09322b8b9792 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:04.899845 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6f98a3-2562-47d2-b345-09322b8b9792 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:43:04.900154 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6f98a3-2562-47d2-b345-09322b8b9792 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:04.901752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6f98a3-2562-47d2-b345-09322b8b9792 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:04.913147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1337/2669] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:43:04.917771 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd061627-386f-4ceb-90d5-8f9e2be70b88 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:04.928074 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd061627-386f-4ceb-90d5-8f9e2be70b88 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:04.932745 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7791704-45a8-4a37-802b-193d8b23ce25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:04.933391 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7791704-45a8-4a37-802b-193d8b23ce25 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:04.933674 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7791704-45a8-4a37-802b-193d8b23ce25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:04.933950 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7791704-45a8-4a37-802b-193d8b23ce25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:04.934676 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7791704-45a8-4a37-802b-193d8b23ce25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:04.936695 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7791704-45a8-4a37-802b-193d8b23ce25 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:05.071704 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1333/2670] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:43:04 2022] POST /identity/v3/projects => generated 383 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:43:05.076036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f98a3-2562-47d2-b345-09322b8b9792 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:05.090414 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f98a3-2562-47d2-b345-09322b8b9792 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:05.148101 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4056490-fe3b-4926-8103-28b422c83d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:05.148802 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4056490-fe3b-4926-8103-28b422c83d5c admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:05.149088 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4056490-fe3b-4926-8103-28b422c83d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:05.149368 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4056490-fe3b-4926-8103-28b422c83d5c admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:05.150063 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4056490-fe3b-4926-8103-28b422c83d5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:05.152022 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4056490-fe3b-4926-8103-28b422c83d5c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:05.159576 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:43:05.257688 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1338/2671] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:43:05 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:43:05.261810 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7791704-45a8-4a37-802b-193d8b23ce25 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:05.272196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7791704-45a8-4a37-802b-193d8b23ce25 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:05.277269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac398ed-d98f-4d1b-bbe1-0c19e2e1b471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:05.278135 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac398ed-d98f-4d1b-bbe1-0c19e2e1b471 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:05.278421 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac398ed-d98f-4d1b-bbe1-0c19e2e1b471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:05.278707 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac398ed-d98f-4d1b-bbe1-0c19e2e1b471 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:05.279499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac398ed-d98f-4d1b-bbe1-0c19e2e1b471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:05.289302 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac398ed-d98f-4d1b-bbe1-0c19e2e1b471 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:05.292675 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1334/2672] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:43:05.296474 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4056490-fe3b-4926-8103-28b422c83d5c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:05.307312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4056490-fe3b-4926-8103-28b422c83d5c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:05.312092 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40304e44-2244-4eec-b592-08a75e4b0dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:05.312755 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40304e44-2244-4eec-b592-08a75e4b0dd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:05.313040 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40304e44-2244-4eec-b592-08a75e4b0dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:05.313361 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40304e44-2244-4eec-b592-08a75e4b0dd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:05.314081 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40304e44-2244-4eec-b592-08a75e4b0dd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:05.315764 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40304e44-2244-4eec-b592-08a75e4b0dd8 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:05.326413 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1339/2673] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:43:05.330613 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac398ed-d98f-4d1b-bbe1-0c19e2e1b471 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:05.340804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac398ed-d98f-4d1b-bbe1-0c19e2e1b471 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:05.345409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a260376-0791-4e85-b1d5-afeb3d398c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:05.346274 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a260376-0791-4e85-b1d5-afeb3d398c86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:05.346551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a260376-0791-4e85-b1d5-afeb3d398c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:05.346821 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a260376-0791-4e85-b1d5-afeb3d398c86 admin admin] PATH_INFO: `/v3/projects/373fdedbc63a4f149d32b610ca0611e3/users/9c8d2fbb45474d33b93bcdddfd79518f/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:05.361814 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a260376-0791-4e85-b1d5-afeb3d398c86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=373fdedbc63a4f149d32b610ca0611e3, user_id=9c8d2fbb45474d33b93bcdddfd79518f, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:05.365439 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a260376-0791-4e85-b1d5-afeb3d398c86 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:05.502802 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1335/2674] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:43:05 2022] PUT /identity/v3/projects/373fdedbc63a4f149d32b610ca0611e3/users/9c8d2fbb45474d33b93bcdddfd79518f/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:43:05.513155 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453c728-09fe-4aec-961a-e7978f99e05b None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:05.513458 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453c728-09fe-4aec-961a-e7978f99e05b None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:05.513717 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453c728-09fe-4aec-961a-e7978f99e05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:05.546486 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-f453c728-09fe-4aec-961a-e7978f99e05b None None] MFA Rules not processed for user `9c8d2fbb45474d33b93bcdddfd79518f`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:05.548619 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f453c728-09fe-4aec-961a-e7978f99e05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:05.614666 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1340/2675] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:43:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:05.619226 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a260376-0791-4e85-b1d5-afeb3d398c86 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:05.679380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a260376-0791-4e85-b1d5-afeb3d398c86 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:05.684108 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1029c-a1ae-4396-980d-81471cf3e7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:05.684756 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1029c-a1ae-4396-980d-81471cf3e7f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:05.685034 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1029c-a1ae-4396-980d-81471cf3e7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:05.685306 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1029c-a1ae-4396-980d-81471cf3e7f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:05.686000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1029c-a1ae-4396-980d-81471cf3e7f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:05.687934 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1029c-a1ae-4396-980d-81471cf3e7f1 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:05.704759 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:43:05.789847 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1336/2676] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:43:05 2022] POST /identity/v3/projects => generated 381 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:43:05.791083 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453c728-09fe-4aec-961a-e7978f99e05b None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:05.801725 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453c728-09fe-4aec-961a-e7978f99e05b None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:05.857978 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a0d86-9a0e-4059-b47c-e09fd9ca5e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:05.858659 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a0d86-9a0e-4059-b47c-e09fd9ca5e76 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:05.858979 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a0d86-9a0e-4059-b47c-e09fd9ca5e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:05.859262 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a0d86-9a0e-4059-b47c-e09fd9ca5e76 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:05.859959 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a0d86-9a0e-4059-b47c-e09fd9ca5e76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:05.862080 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a0d86-9a0e-4059-b47c-e09fd9ca5e76 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:05.983417 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1341/2677] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:43:05 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:43:05.987562 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1029c-a1ae-4396-980d-81471cf3e7f1 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:05.997940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1029c-a1ae-4396-980d-81471cf3e7f1 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:06.002589 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d38823-4276-49f2-b29d-6eb75ed126a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:06.003350 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d38823-4276-49f2-b29d-6eb75ed126a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:06.003635 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d38823-4276-49f2-b29d-6eb75ed126a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:06.003922 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d38823-4276-49f2-b29d-6eb75ed126a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:06.005127 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d38823-4276-49f2-b29d-6eb75ed126a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:06.006725 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d38823-4276-49f2-b29d-6eb75ed126a2 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:06.017176 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1337/2678] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:43:06.025785 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a0d86-9a0e-4059-b47c-e09fd9ca5e76 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:06.036058 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a0d86-9a0e-4059-b47c-e09fd9ca5e76 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:06.040975 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d59a3f-1e05-4a2a-bacd-00cecf192b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:06.041843 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d59a3f-1e05-4a2a-bacd-00cecf192b93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:06.042130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d59a3f-1e05-4a2a-bacd-00cecf192b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:06.042409 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d59a3f-1e05-4a2a-bacd-00cecf192b93 admin admin] PATH_INFO: `/v3/projects/060974fd756948a2bd15c334dd01a329/users/d56b8ed53130443f8b8a34a0658f9f25/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:06.057364 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d59a3f-1e05-4a2a-bacd-00cecf192b93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=060974fd756948a2bd15c334dd01a329, user_id=d56b8ed53130443f8b8a34a0658f9f25, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:06.060399 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d59a3f-1e05-4a2a-bacd-00cecf192b93 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:06.191126 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1342/2679] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:43:06 2022] PUT /identity/v3/projects/060974fd756948a2bd15c334dd01a329/users/d56b8ed53130443f8b8a34a0658f9f25/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:43:06.195555 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d38823-4276-49f2-b29d-6eb75ed126a2 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:06.206426 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d38823-4276-49f2-b29d-6eb75ed126a2 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:06.266110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3653b-dc7f-4a35-b379-b77ad6415899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:06.266729 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3653b-dc7f-4a35-b379-b77ad6415899 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:06.267043 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3653b-dc7f-4a35-b379-b77ad6415899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:06.267328 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3653b-dc7f-4a35-b379-b77ad6415899 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:06.268771 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc3653b-dc7f-4a35-b379-b77ad6415899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:06.271702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc3653b-dc7f-4a35-b379-b77ad6415899 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:06.285468 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1338/2680] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:43:06.289497 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d59a3f-1e05-4a2a-bacd-00cecf192b93 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:06.301938 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d59a3f-1e05-4a2a-bacd-00cecf192b93 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:06.308016 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6244e13-70ce-4279-b737-0b082572eefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:06.308933 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6244e13-70ce-4279-b737-0b082572eefc admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:06.309289 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6244e13-70ce-4279-b737-0b082572eefc admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:06.309616 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6244e13-70ce-4279-b737-0b082572eefc admin admin] PATH_INFO: `/v3/projects/060974fd756948a2bd15c334dd01a329/users/d56b8ed53130443f8b8a34a0658f9f25/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:06.312070 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6244e13-70ce-4279-b737-0b082572eefc admin admin] RBAC: Authorizing `identity:create_grant(project_id=060974fd756948a2bd15c334dd01a329, user_id=d56b8ed53130443f8b8a34a0658f9f25, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:06.314497 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6244e13-70ce-4279-b737-0b082572eefc admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:06.325782 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1343/2681] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:43:06 2022] PUT /identity/v3/projects/060974fd756948a2bd15c334dd01a329/users/d56b8ed53130443f8b8a34a0658f9f25/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:43:06.337484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d8fa9-bd0e-41db-b266-38f61d3a94a6 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:06.337775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d8fa9-bd0e-41db-b266-38f61d3a94a6 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:06.338238 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d8fa9-bd0e-41db-b266-38f61d3a94a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:06.364954 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-1f1d8fa9-bd0e-41db-b266-38f61d3a94a6 None None] MFA Rules not processed for user `d56b8ed53130443f8b8a34a0658f9f25`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:06.366151 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1f1d8fa9-bd0e-41db-b266-38f61d3a94a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:06.433260 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1339/2682] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:43:06 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:06.445274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6244e13-70ce-4279-b737-0b082572eefc admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:06.447002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b6244e13-70ce-4279-b737-0b082572eefc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:06.473450 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:43:06.505125 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6244e13-70ce-4279-b737-0b082572eefc admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:06.508745 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6f257-8d38-4767-b66c-ab4ecccdf8a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:06.509382 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6f257-8d38-4767-b66c-ab4ecccdf8a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:06.509659 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6f257-8d38-4767-b66c-ab4ecccdf8a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:06.509929 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6f257-8d38-4767-b66c-ab4ecccdf8a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:06.518616 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6f257-8d38-4767-b66c-ab4ecccdf8a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:06.520068 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af6f257-8d38-4767-b66c-ab4ecccdf8a7 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:06.529255 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1344/2683] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:43:06 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:43:06.687268 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d8fa9-bd0e-41db-b266-38f61d3a94a6 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:06.699774 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d8fa9-bd0e-41db-b266-38f61d3a94a6 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:06.703575 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a6fe00-b152-4b65-b039-aa03bdedfb34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:06.704226 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6fe00-b152-4b65-b039-aa03bdedfb34 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:06.704504 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6fe00-b152-4b65-b039-aa03bdedfb34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:06.704779 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6fe00-b152-4b65-b039-aa03bdedfb34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:06.762537 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a6fe00-b152-4b65-b039-aa03bdedfb34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:06.764021 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a6fe00-b152-4b65-b039-aa03bdedfb34 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:06.772988 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1340/2684] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:43:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:43:07.260164 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6f257-8d38-4767-b66c-ab4ecccdf8a7 service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:07.325480 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6f257-8d38-4767-b66c-ab4ecccdf8a7 service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:07.330430 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d55dc-85c1-420f-8070-f7b09a8b5ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:07.331888 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d55dc-85c1-420f-8070-f7b09a8b5ab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:07.331888 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d55dc-85c1-420f-8070-f7b09a8b5ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:07.332099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d55dc-85c1-420f-8070-f7b09a8b5ab4 admin admin] PATH_INFO: `/v3/users/9c8d2fbb45474d33b93bcdddfd79518f` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:07.334052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d55dc-85c1-420f-8070-f7b09a8b5ab4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c8d2fbb45474d33b93bcdddfd79518f)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:07.336014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086d55dc-85c1-420f-8070-f7b09a8b5ab4 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:07.552236 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-086d55dc-85c1-420f-8070-f7b09a8b5ab4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c8d2fbb45474d33b93bcdddfd79518f'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:07.663227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-086d55dc-85c1-420f-8070-f7b09a8b5ab4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c8d2fbb45474d33b93bcdddfd79518f'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:07.677310 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-086d55dc-85c1-420f-8070-f7b09a8b5ab4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c8d2fbb45474d33b93bcdddfd79518f'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:07.678017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-086d55dc-85c1-420f-8070-f7b09a8b5ab4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c8d2fbb45474d33b93bcdddfd79518f'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:07.685183 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-086d55dc-85c1-420f-8070-f7b09a8b5ab4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c8d2fbb45474d33b93bcdddfd79518f'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:07.694165 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1345/2685] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:43:07 2022] DELETE /identity/v3/users/9c8d2fbb45474d33b93bcdddfd79518f => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:43:07.699390 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb1426-4e13-4395-b6e2-9dbb90348db5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:07.700612 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb1426-4e13-4395-b6e2-9dbb90348db5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:07.700967 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb1426-4e13-4395-b6e2-9dbb90348db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:07.718538 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:43:07.740784 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-3bcb1426-4e13-4395-b6e2-9dbb90348db5 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:07.743145 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3bcb1426-4e13-4395-b6e2-9dbb90348db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:07.826110 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1341/2686] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:43:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:07.838420 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d55dc-85c1-420f-8070-f7b09a8b5ab4 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:07.839953 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-086d55dc-85c1-420f-8070-f7b09a8b5ab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:07.899366 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:43:07.909830 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086d55dc-85c1-420f-8070-f7b09a8b5ab4 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:07.913480 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be40da-1bb3-4835-b0f9-c60c333b863f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:07.914125 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be40da-1bb3-4835-b0f9-c60c333b863f service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:07.914402 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be40da-1bb3-4835-b0f9-c60c333b863f service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:07.914679 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be40da-1bb3-4835-b0f9-c60c333b863f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:07.923865 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be40da-1bb3-4835-b0f9-c60c333b863f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:07.925330 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be40da-1bb3-4835-b0f9-c60c333b863f service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:07.935632 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1346/2687] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:43:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:43:08.414428 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb1426-4e13-4395-b6e2-9dbb90348db5 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:08.419730 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3bcb1426-4e13-4395-b6e2-9dbb90348db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:08.436306 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb1426-4e13-4395-b6e2-9dbb90348db5 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:08.441795 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a93aa-df80-4af9-82c5-a5cef148bd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:08.442685 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a93aa-df80-4af9-82c5-a5cef148bd73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:08.442996 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a93aa-df80-4af9-82c5-a5cef148bd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:08.443266 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a93aa-df80-4af9-82c5-a5cef148bd73 admin admin] PATH_INFO: `/v3/projects/373fdedbc63a4f149d32b610ca0611e3` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:08.444476 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a93aa-df80-4af9-82c5-a5cef148bd73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=373fdedbc63a4f149d32b610ca0611e3)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:08.446522 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a93aa-df80-4af9-82c5-a5cef148bd73 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:08.602497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ad1a93aa-df80-4af9-82c5-a5cef148bd73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '373fdedbc63a4f149d32b610ca0611e3', 'request_id': 'req-ad1a93aa-df80-4af9-82c5-a5cef148bd73'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:08.709957 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ad1a93aa-df80-4af9-82c5-a5cef148bd73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '373fdedbc63a4f149d32b610ca0611e3', 'request_id': 'req-ad1a93aa-df80-4af9-82c5-a5cef148bd73'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:08.718906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ad1a93aa-df80-4af9-82c5-a5cef148bd73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '373fdedbc63a4f149d32b610ca0611e3', 'request_id': 'req-ad1a93aa-df80-4af9-82c5-a5cef148bd73'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:08.733243 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ad1a93aa-df80-4af9-82c5-a5cef148bd73 admin admin] The token cache is being invalidate because project 373fdedbc63a4f149d32b610ca0611e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:43:08.733545 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ad1a93aa-df80-4af9-82c5-a5cef148bd73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:08.734348 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ad1a93aa-df80-4af9-82c5-a5cef148bd73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:08.736684 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1342/2688] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:43:08 2022] DELETE /identity/v3/projects/373fdedbc63a4f149d32b610ca0611e3 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:43:08.740684 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be40da-1bb3-4835-b0f9-c60c333b863f service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:08.751607 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be40da-1bb3-4835-b0f9-c60c333b863f service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:08.810575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df751476-fcae-41f5-b422-7f256b46f4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:08.811485 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df751476-fcae-41f5-b422-7f256b46f4f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:08.811759 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df751476-fcae-41f5-b422-7f256b46f4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:08.812027 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df751476-fcae-41f5-b422-7f256b46f4f5 admin admin] PATH_INFO: `/v3/users/d56b8ed53130443f8b8a34a0658f9f25` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:08.813215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df751476-fcae-41f5-b422-7f256b46f4f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d56b8ed53130443f8b8a34a0658f9f25)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:08.815209 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df751476-fcae-41f5-b422-7f256b46f4f5 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:09.087730 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-df751476-fcae-41f5-b422-7f256b46f4f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd56b8ed53130443f8b8a34a0658f9f25'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:09.160579 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-df751476-fcae-41f5-b422-7f256b46f4f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd56b8ed53130443f8b8a34a0658f9f25'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:09.173944 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-df751476-fcae-41f5-b422-7f256b46f4f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd56b8ed53130443f8b8a34a0658f9f25'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:09.174738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-df751476-fcae-41f5-b422-7f256b46f4f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd56b8ed53130443f8b8a34a0658f9f25'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:09.184671 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-df751476-fcae-41f5-b422-7f256b46f4f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd56b8ed53130443f8b8a34a0658f9f25'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:09.191508 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1347/2689] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:43:08 2022] DELETE /identity/v3/users/d56b8ed53130443f8b8a34a0658f9f25 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:43:09.778994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a93aa-df80-4af9-82c5-a5cef148bd73 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:09.778994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ad1a93aa-df80-4af9-82c5-a5cef148bd73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:09.792155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a93aa-df80-4af9-82c5-a5cef148bd73 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:09.855580 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574bf850-7e5a-4a6a-a6c6-9edba3d04f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:09.856450 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bf850-7e5a-4a6a-a6c6-9edba3d04f48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:09.856733 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bf850-7e5a-4a6a-a6c6-9edba3d04f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:09.857009 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bf850-7e5a-4a6a-a6c6-9edba3d04f48 admin admin] PATH_INFO: `/v3/projects/060974fd756948a2bd15c334dd01a329` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:09.858854 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574bf850-7e5a-4a6a-a6c6-9edba3d04f48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=060974fd756948a2bd15c334dd01a329)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:09.860802 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574bf850-7e5a-4a6a-a6c6-9edba3d04f48 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:10.062503 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:43:10.081902 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-574bf850-7e5a-4a6a-a6c6-9edba3d04f48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '060974fd756948a2bd15c334dd01a329', 'request_id': 'req-574bf850-7e5a-4a6a-a6c6-9edba3d04f48'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:10.189554 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-574bf850-7e5a-4a6a-a6c6-9edba3d04f48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '060974fd756948a2bd15c334dd01a329', 'request_id': 'req-574bf850-7e5a-4a6a-a6c6-9edba3d04f48'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:10.197671 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-574bf850-7e5a-4a6a-a6c6-9edba3d04f48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '060974fd756948a2bd15c334dd01a329', 'request_id': 'req-574bf850-7e5a-4a6a-a6c6-9edba3d04f48'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:10.211241 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-574bf850-7e5a-4a6a-a6c6-9edba3d04f48 admin admin] The token cache is being invalidate because project 060974fd756948a2bd15c334dd01a329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:43:10.211579 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-574bf850-7e5a-4a6a-a6c6-9edba3d04f48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:10.212186 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-574bf850-7e5a-4a6a-a6c6-9edba3d04f48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:10.213771 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1343/2690] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:43:09 2022] DELETE /identity/v3/projects/060974fd756948a2bd15c334dd01a329 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:43:10.218624 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ec9f8-cbb8-4d38-8c32-df7908adb790 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:10.219163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ec9f8-cbb8-4d38-8c32-df7908adb790 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:10.219282 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6ec9f8-cbb8-4d38-8c32-df7908adb790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:10.240247 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-0e6ec9f8-cbb8-4d38-8c32-df7908adb790 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:10.241486 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-0e6ec9f8-cbb8-4d38-8c32-df7908adb790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:10.302697 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1348/2691] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:43:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:10.315183 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aba313-37c2-4795-b8ed-e1466c544092 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:10.315472 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aba313-37c2-4795-b8ed-e1466c544092 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:10.315746 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aba313-37c2-4795-b8ed-e1466c544092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:10.336332 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-79aba313-37c2-4795-b8ed-e1466c544092 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:10.342004 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-79aba313-37c2-4795-b8ed-e1466c544092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:10.345803 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1344/2692] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:43:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:10.350670 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6ec9f8-cbb8-4d38-8c32-df7908adb790 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:10.360257 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6ec9f8-cbb8-4d38-8c32-df7908adb790 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:10.365622 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8621f-a616-4e41-92f9-97f5f32decaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:10.366301 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8621f-a616-4e41-92f9-97f5f32decaf admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:10.366615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8621f-a616-4e41-92f9-97f5f32decaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:10.366947 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8621f-a616-4e41-92f9-97f5f32decaf admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:10.367715 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e8621f-a616-4e41-92f9-97f5f32decaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:43:10.368026 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e8621f-a616-4e41-92f9-97f5f32decaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:10.369704 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e8621f-a616-4e41-92f9-97f5f32decaf admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:10.371333 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:43:10.382212 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1349/2693] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:43:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:43:10.386489 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aba313-37c2-4795-b8ed-e1466c544092 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:10.395721 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aba313-37c2-4795-b8ed-e1466c544092 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:10.400357 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0283a79-1ede-43bd-879e-8b1b2542298a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:10.401071 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0283a79-1ede-43bd-879e-8b1b2542298a admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:10.401352 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0283a79-1ede-43bd-879e-8b1b2542298a admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:10.401625 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0283a79-1ede-43bd-879e-8b1b2542298a admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:10.402307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0283a79-1ede-43bd-879e-8b1b2542298a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:10.404218 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0283a79-1ede-43bd-879e-8b1b2542298a admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:10.470464 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1345/2694] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:43:10 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:43:10.475372 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8621f-a616-4e41-92f9-97f5f32decaf admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:10.485689 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8621f-a616-4e41-92f9-97f5f32decaf admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:10.541014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1021c3b1-fbfc-4b2f-9637-e9cfb75bba8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:10.541685 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021c3b1-fbfc-4b2f-9637-e9cfb75bba8c admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:10.541969 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021c3b1-fbfc-4b2f-9637-e9cfb75bba8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:10.542245 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021c3b1-fbfc-4b2f-9637-e9cfb75bba8c admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:10.543089 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1021c3b1-fbfc-4b2f-9637-e9cfb75bba8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:10.545012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1021c3b1-fbfc-4b2f-9637-e9cfb75bba8c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:10.640684 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1350/2695] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:43:10 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:43:10.644729 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0283a79-1ede-43bd-879e-8b1b2542298a admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:10.655631 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0283a79-1ede-43bd-879e-8b1b2542298a admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:10.660422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a4c1a-8019-459c-ae9c-62ad76b6473b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:10.661188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a4c1a-8019-459c-ae9c-62ad76b6473b admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:10.661468 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a4c1a-8019-459c-ae9c-62ad76b6473b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:10.661740 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a4c1a-8019-459c-ae9c-62ad76b6473b admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:10.662466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a4c1a-8019-459c-ae9c-62ad76b6473b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:10.664099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a4c1a-8019-459c-ae9c-62ad76b6473b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:10.673875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1346/2696] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:43:10.677789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1021c3b1-fbfc-4b2f-9637-e9cfb75bba8c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:10.687622 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1021c3b1-fbfc-4b2f-9637-e9cfb75bba8c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:10.692994 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15bd03-1354-4b71-a6ae-b974c8237faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:10.693637 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15bd03-1354-4b71-a6ae-b974c8237faf admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:10.693915 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15bd03-1354-4b71-a6ae-b974c8237faf admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:10.694186 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15bd03-1354-4b71-a6ae-b974c8237faf admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:10.694924 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c15bd03-1354-4b71-a6ae-b974c8237faf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:10.696533 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c15bd03-1354-4b71-a6ae-b974c8237faf admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:10.706319 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1351/2697] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:43:10.711021 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a4c1a-8019-459c-ae9c-62ad76b6473b admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:10.721001 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a4c1a-8019-459c-ae9c-62ad76b6473b admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:10.725711 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ed86f-1b41-4bc8-a285-eec49ae6caa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:10.726582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ed86f-1b41-4bc8-a285-eec49ae6caa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:10.726897 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ed86f-1b41-4bc8-a285-eec49ae6caa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:10.727180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ed86f-1b41-4bc8-a285-eec49ae6caa2 admin admin] PATH_INFO: `/v3/projects/3a6e85b9011c4bd3aab9682beefe5d95/users/c5bccc463c584433aa4c3aaa3b9aeccd/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:10.741974 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ed86f-1b41-4bc8-a285-eec49ae6caa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a6e85b9011c4bd3aab9682beefe5d95, user_id=c5bccc463c584433aa4c3aaa3b9aeccd, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:10.744650 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ed86f-1b41-4bc8-a285-eec49ae6caa2 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:10.863258 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1347/2698] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:43:10 2022] PUT /identity/v3/projects/3a6e85b9011c4bd3aab9682beefe5d95/users/c5bccc463c584433aa4c3aaa3b9aeccd/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:43:10.874955 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c28d4-f952-4074-b25b-1ae4983e6836 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:10.875251 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c28d4-f952-4074-b25b-1ae4983e6836 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:10.875528 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c28d4-f952-4074-b25b-1ae4983e6836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:10.902628 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-9b1c28d4-f952-4074-b25b-1ae4983e6836 None None] MFA Rules not processed for user `c5bccc463c584433aa4c3aaa3b9aeccd`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:10.903855 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-9b1c28d4-f952-4074-b25b-1ae4983e6836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:10.966201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1352/2699] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:43:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:10.970687 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ed86f-1b41-4bc8-a285-eec49ae6caa2 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:11.030903 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ed86f-1b41-4bc8-a285-eec49ae6caa2 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:11.035605 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94d3f9-45e3-4afb-9377-9be6e50ed35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:11.036251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94d3f9-45e3-4afb-9377-9be6e50ed35c admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:11.036527 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94d3f9-45e3-4afb-9377-9be6e50ed35c admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:11.036796 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc94d3f9-45e3-4afb-9377-9be6e50ed35c admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:11.037484 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc94d3f9-45e3-4afb-9377-9be6e50ed35c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:11.039411 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc94d3f9-45e3-4afb-9377-9be6e50ed35c admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:11.139226 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1348/2700] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:43:10 2022] POST /identity/v3/projects => generated 385 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:43:11.143187 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c28d4-f952-4074-b25b-1ae4983e6836 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:11.155168 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c28d4-f952-4074-b25b-1ae4983e6836 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:11.211257 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd54fe9-cad4-43dc-a4cc-1bbba1f900c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:11.211913 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd54fe9-cad4-43dc-a4cc-1bbba1f900c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:11.212192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd54fe9-cad4-43dc-a4cc-1bbba1f900c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:11.212464 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd54fe9-cad4-43dc-a4cc-1bbba1f900c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:11.213143 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd54fe9-cad4-43dc-a4cc-1bbba1f900c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:11.215066 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd54fe9-cad4-43dc-a4cc-1bbba1f900c5 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:11.222407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:43:11.348820 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1353/2701] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:43:11 2022] POST /identity/v3/users => generated 293 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:43:11.349998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94d3f9-45e3-4afb-9377-9be6e50ed35c admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:11.361706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc94d3f9-45e3-4afb-9377-9be6e50ed35c admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:11.367365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e7f41-4829-4fac-b945-c81cb16a90e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:11.368038 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e7f41-4829-4fac-b945-c81cb16a90e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:11.368322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e7f41-4829-4fac-b945-c81cb16a90e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:11.368620 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3e7f41-4829-4fac-b945-c81cb16a90e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:11.369320 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3e7f41-4829-4fac-b945-c81cb16a90e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:11.371816 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3e7f41-4829-4fac-b945-c81cb16a90e8 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:11.383136 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1349/2702] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:43:11.387253 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd54fe9-cad4-43dc-a4cc-1bbba1f900c5 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:11.397396 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd54fe9-cad4-43dc-a4cc-1bbba1f900c5 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:11.403448 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a90f9-648b-4f66-90d0-ff2c4bbea728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:11.404199 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a90f9-648b-4f66-90d0-ff2c4bbea728 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:11.404481 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a90f9-648b-4f66-90d0-ff2c4bbea728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:11.404815 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a90f9-648b-4f66-90d0-ff2c4bbea728 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:11.405581 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a90f9-648b-4f66-90d0-ff2c4bbea728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:11.407288 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a90f9-648b-4f66-90d0-ff2c4bbea728 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:11.421313 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1354/2703] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:43:11.423344 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e7f41-4829-4fac-b945-c81cb16a90e8 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:11.433637 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3e7f41-4829-4fac-b945-c81cb16a90e8 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:11.439428 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07701389-66af-422e-a9c9-9329b52d27a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:11.440671 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07701389-66af-422e-a9c9-9329b52d27a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:11.441031 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07701389-66af-422e-a9c9-9329b52d27a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:11.441549 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07701389-66af-422e-a9c9-9329b52d27a7 admin admin] PATH_INFO: `/v3/projects/e0fffadff6764fabbcba0ff1fa6a0383/users/35f8a86ab3c241d98def02d2b2a6db6c/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:11.462176 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07701389-66af-422e-a9c9-9329b52d27a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0fffadff6764fabbcba0ff1fa6a0383, user_id=35f8a86ab3c241d98def02d2b2a6db6c, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:11.464876 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07701389-66af-422e-a9c9-9329b52d27a7 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:11.560571 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1350/2704] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:43:11 2022] PUT /identity/v3/projects/e0fffadff6764fabbcba0ff1fa6a0383/users/35f8a86ab3c241d98def02d2b2a6db6c/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:43:11.570724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a22921-99e7-44e2-8cba-8d66252731c2 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:11.571180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a22921-99e7-44e2-8cba-8d66252731c2 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:11.571471 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a22921-99e7-44e2-8cba-8d66252731c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:11.598633 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-80a22921-99e7-44e2-8cba-8d66252731c2 None None] MFA Rules not processed for user `35f8a86ab3c241d98def02d2b2a6db6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:11.599815 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-80a22921-99e7-44e2-8cba-8d66252731c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:11.663243 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1355/2705] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:43:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:11.667729 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07701389-66af-422e-a9c9-9329b52d27a7 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:11.673375 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:43:11.729749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07701389-66af-422e-a9c9-9329b52d27a7 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:11.734426 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c6dc9-3e7e-4986-bf6a-9474f78fb37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:11.735164 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c6dc9-3e7e-4986-bf6a-9474f78fb37c admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:11.735454 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c6dc9-3e7e-4986-bf6a-9474f78fb37c admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:11.735732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c6dc9-3e7e-4986-bf6a-9474f78fb37c admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:11.736439 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0c6dc9-3e7e-4986-bf6a-9474f78fb37c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:11.738351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0c6dc9-3e7e-4986-bf6a-9474f78fb37c admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:11.837693 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1351/2706] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:43:11 2022] POST /identity/v3/projects => generated 385 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:43:11.841752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a22921-99e7-44e2-8cba-8d66252731c2 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:11.853024 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a22921-99e7-44e2-8cba-8d66252731c2 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:11.911845 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a42f8-312b-411e-af1e-f1e345d43c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:11.912518 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a42f8-312b-411e-af1e-f1e345d43c2d admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:11.912802 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a42f8-312b-411e-af1e-f1e345d43c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:11.913081 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280a42f8-312b-411e-af1e-f1e345d43c2d admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:11.913823 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280a42f8-312b-411e-af1e-f1e345d43c2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:11.915746 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280a42f8-312b-411e-af1e-f1e345d43c2d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:12.012728 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1356/2707] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:43:11 2022] POST /identity/v3/users => generated 299 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:43:12.016573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c6dc9-3e7e-4986-bf6a-9474f78fb37c admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:12.027434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c6dc9-3e7e-4986-bf6a-9474f78fb37c admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:12.032002 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e9fe0-4706-4f15-8e12-0bbc8b588b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:12.032647 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e9fe0-4706-4f15-8e12-0bbc8b588b7f admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:12.032959 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e9fe0-4706-4f15-8e12-0bbc8b588b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:12.033236 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e9fe0-4706-4f15-8e12-0bbc8b588b7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:12.033964 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e9fe0-4706-4f15-8e12-0bbc8b588b7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:12.035600 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e9fe0-4706-4f15-8e12-0bbc8b588b7f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:12.046005 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1352/2708] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:43:12.050019 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a42f8-312b-411e-af1e-f1e345d43c2d admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:12.060076 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280a42f8-312b-411e-af1e-f1e345d43c2d admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:12.064708 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739c47a-bfae-4228-96b7-637eb2dcf224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:12.065603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739c47a-bfae-4228-96b7-637eb2dcf224 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:12.065869 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739c47a-bfae-4228-96b7-637eb2dcf224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:12.066148 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739c47a-bfae-4228-96b7-637eb2dcf224 admin admin] PATH_INFO: `/v3/projects/cee3ad8d60154e0bab4a0c16c63e8e83/users/80262ec36b5447d5baf4d6ddaa21a525/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:12.084822 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d739c47a-bfae-4228-96b7-637eb2dcf224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cee3ad8d60154e0bab4a0c16c63e8e83, user_id=80262ec36b5447d5baf4d6ddaa21a525, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:12.087507 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d739c47a-bfae-4228-96b7-637eb2dcf224 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:12.160889 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1357/2709] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:43:12 2022] PUT /identity/v3/projects/cee3ad8d60154e0bab4a0c16c63e8e83/users/80262ec36b5447d5baf4d6ddaa21a525/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:43:12.165071 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e9fe0-4706-4f15-8e12-0bbc8b588b7f admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:12.175265 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e9fe0-4706-4f15-8e12-0bbc8b588b7f admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:12.235281 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8275aaa-2dd3-4168-a0c7-8a2b0e79901e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:12.235953 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8275aaa-2dd3-4168-a0c7-8a2b0e79901e admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:12.236238 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8275aaa-2dd3-4168-a0c7-8a2b0e79901e admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:12.236547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8275aaa-2dd3-4168-a0c7-8a2b0e79901e admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:12.237314 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8275aaa-2dd3-4168-a0c7-8a2b0e79901e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:12.239508 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8275aaa-2dd3-4168-a0c7-8a2b0e79901e admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:12.249707 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1353/2710] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:43:12.253888 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739c47a-bfae-4228-96b7-637eb2dcf224 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:12.264979 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d739c47a-bfae-4228-96b7-637eb2dcf224 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:12.269881 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08b05c-129b-405a-ac62-c87b53e733be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:12.270775 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08b05c-129b-405a-ac62-c87b53e733be admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:12.271082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08b05c-129b-405a-ac62-c87b53e733be admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:12.271362 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08b05c-129b-405a-ac62-c87b53e733be admin admin] PATH_INFO: `/v3/projects/cee3ad8d60154e0bab4a0c16c63e8e83/users/80262ec36b5447d5baf4d6ddaa21a525/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:12.273316 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab08b05c-129b-405a-ac62-c87b53e733be admin admin] RBAC: Authorizing `identity:create_grant(project_id=cee3ad8d60154e0bab4a0c16c63e8e83, user_id=80262ec36b5447d5baf4d6ddaa21a525, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:12.276886 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab08b05c-129b-405a-ac62-c87b53e733be admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:12.286791 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1358/2711] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:43:12 2022] PUT /identity/v3/projects/cee3ad8d60154e0bab4a0c16c63e8e83/users/80262ec36b5447d5baf4d6ddaa21a525/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:43:12.296940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de3aed-c57f-49a3-a65d-2ea1667aa94a None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:12.297232 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de3aed-c57f-49a3-a65d-2ea1667aa94a None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:12.299894 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de3aed-c57f-49a3-a65d-2ea1667aa94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:12.324030 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d6de3aed-c57f-49a3-a65d-2ea1667aa94a None None] MFA Rules not processed for user `80262ec36b5447d5baf4d6ddaa21a525`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:12.325214 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d6de3aed-c57f-49a3-a65d-2ea1667aa94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:12.389169 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:43:12.393069 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1354/2712] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:43:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:12.404366 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08b05c-129b-405a-ac62-c87b53e733be admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:12.406076 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ab08b05c-129b-405a-ac62-c87b53e733be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:12.433231 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:43:12.463674 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08b05c-129b-405a-ac62-c87b53e733be admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:12.467244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f55eed-8a8c-47a7-8273-93a8dbf420b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:12.467888 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f55eed-8a8c-47a7-8273-93a8dbf420b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:12.468166 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f55eed-8a8c-47a7-8273-93a8dbf420b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:12.468444 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f55eed-8a8c-47a7-8273-93a8dbf420b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:12.526329 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f55eed-8a8c-47a7-8273-93a8dbf420b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:12.527937 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f55eed-8a8c-47a7-8273-93a8dbf420b2 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:12.538956 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1359/2713] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:43:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:43:14.589773 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de3aed-c57f-49a3-a65d-2ea1667aa94a None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:14.591240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d6de3aed-c57f-49a3-a65d-2ea1667aa94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:14.605845 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de3aed-c57f-49a3-a65d-2ea1667aa94a None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:14.609983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b4652-2386-4617-a787-92a9a26b1acc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:14.610633 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b4652-2386-4617-a787-92a9a26b1acc service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:14.610948 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b4652-2386-4617-a787-92a9a26b1acc service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:14.611220 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b4652-2386-4617-a787-92a9a26b1acc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:14.666427 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b4652-2386-4617-a787-92a9a26b1acc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:14.667909 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b4652-2386-4617-a787-92a9a26b1acc service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:14.677351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1355/2714] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:43:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:43:16.490966 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f55eed-8a8c-47a7-8273-93a8dbf420b2 service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:16.499637 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f55eed-8a8c-47a7-8273-93a8dbf420b2 service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:16.507600 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da0ac3-588c-4828-a2c2-2d2909d57baa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:16.507600 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da0ac3-588c-4828-a2c2-2d2909d57baa service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:16.507600 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da0ac3-588c-4828-a2c2-2d2909d57baa service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:16.507600 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da0ac3-588c-4828-a2c2-2d2909d57baa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:16.513973 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da0ac3-588c-4828-a2c2-2d2909d57baa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:16.515472 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da0ac3-588c-4828-a2c2-2d2909d57baa service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:16.525342 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1360/2715] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:43:16 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:43:21.552413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b4652-2386-4617-a787-92a9a26b1acc service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:21.618073 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b4652-2386-4617-a787-92a9a26b1acc service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:21.622984 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3864d1-ba35-4fdc-85db-fab569dcc45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:21.624006 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3864d1-ba35-4fdc-85db-fab569dcc45d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:21.624292 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3864d1-ba35-4fdc-85db-fab569dcc45d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:21.624567 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3864d1-ba35-4fdc-85db-fab569dcc45d admin admin] PATH_INFO: `/v3/users/c5bccc463c584433aa4c3aaa3b9aeccd` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:21.625804 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3864d1-ba35-4fdc-85db-fab569dcc45d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5bccc463c584433aa4c3aaa3b9aeccd)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:21.627787 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3864d1-ba35-4fdc-85db-fab569dcc45d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:21.807887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ba3864d1-ba35-4fdc-85db-fab569dcc45d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5bccc463c584433aa4c3aaa3b9aeccd'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:21.881052 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ba3864d1-ba35-4fdc-85db-fab569dcc45d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5bccc463c584433aa4c3aaa3b9aeccd'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:21.881749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ba3864d1-ba35-4fdc-85db-fab569dcc45d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5bccc463c584433aa4c3aaa3b9aeccd'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:21.889501 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ba3864d1-ba35-4fdc-85db-fab569dcc45d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5bccc463c584433aa4c3aaa3b9aeccd'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:21.890170 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-ba3864d1-ba35-4fdc-85db-fab569dcc45d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5bccc463c584433aa4c3aaa3b9aeccd'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:21.908938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1356/2716] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:43:21 2022] DELETE /identity/v3/users/c5bccc463c584433aa4c3aaa3b9aeccd => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:43:21.913575 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e1f22-3f65-4f43-ab9c-4c10b970be92 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:21.914116 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e1f22-3f65-4f43-ab9c-4c10b970be92 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:21.914116 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e1f22-3f65-4f43-ab9c-4c10b970be92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:21.932437 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-5a9e1f22-3f65-4f43-ab9c-4c10b970be92 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:21.933644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5a9e1f22-3f65-4f43-ab9c-4c10b970be92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:21.995369 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1361/2717] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:43:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:22.005602 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3864d1-ba35-4fdc-85db-fab569dcc45d admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:22.007057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ba3864d1-ba35-4fdc-85db-fab569dcc45d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:22.027144 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:43:22.065202 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3864d1-ba35-4fdc-85db-fab569dcc45d admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:22.069619 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef6e63-a621-408e-92b8-c0d6fa86aa9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:22.070298 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef6e63-a621-408e-92b8-c0d6fa86aa9b service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:22.070679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef6e63-a621-408e-92b8-c0d6fa86aa9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:22.070861 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef6e63-a621-408e-92b8-c0d6fa86aa9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:22.080927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef6e63-a621-408e-92b8-c0d6fa86aa9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:22.082427 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef6e63-a621-408e-92b8-c0d6fa86aa9b service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:22.092549 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1357/2718] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:43:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:43:22.527933 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e1f22-3f65-4f43-ab9c-4c10b970be92 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:22.529370 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5a9e1f22-3f65-4f43-ab9c-4c10b970be92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:22.542632 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e1f22-3f65-4f43-ab9c-4c10b970be92 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:22.547967 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8914616-52b4-4b95-80eb-eb692fc38918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:22.549093 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8914616-52b4-4b95-80eb-eb692fc38918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:22.549441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8914616-52b4-4b95-80eb-eb692fc38918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:22.549783 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8914616-52b4-4b95-80eb-eb692fc38918 admin admin] PATH_INFO: `/v3/projects/3a6e85b9011c4bd3aab9682beefe5d95` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:22.551667 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8914616-52b4-4b95-80eb-eb692fc38918 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a6e85b9011c4bd3aab9682beefe5d95)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:22.553748 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8914616-52b4-4b95-80eb-eb692fc38918 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:22.652347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d8914616-52b4-4b95-80eb-eb692fc38918 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a6e85b9011c4bd3aab9682beefe5d95', 'request_id': 'req-d8914616-52b4-4b95-80eb-eb692fc38918'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:22.714174 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d8914616-52b4-4b95-80eb-eb692fc38918 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a6e85b9011c4bd3aab9682beefe5d95', 'request_id': 'req-d8914616-52b4-4b95-80eb-eb692fc38918'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:22.716573 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:43:22.720774 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d8914616-52b4-4b95-80eb-eb692fc38918 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a6e85b9011c4bd3aab9682beefe5d95', 'request_id': 'req-d8914616-52b4-4b95-80eb-eb692fc38918'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:22.733050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d8914616-52b4-4b95-80eb-eb692fc38918 admin admin] The token cache is being invalidate because project 3a6e85b9011c4bd3aab9682beefe5d95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:43:22.733336 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d8914616-52b4-4b95-80eb-eb692fc38918 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:22.733983 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d8914616-52b4-4b95-80eb-eb692fc38918 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:22.735617 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1362/2719] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:43:22 2022] DELETE /identity/v3/projects/3a6e85b9011c4bd3aab9682beefe5d95 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:43:22.739704 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef6e63-a621-408e-92b8-c0d6fa86aa9b service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:22.749572 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef6e63-a621-408e-92b8-c0d6fa86aa9b service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:22.807101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e39c1-87db-42a9-add1-fc676aee389d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:22.807999 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e39c1-87db-42a9-add1-fc676aee389d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:22.808283 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e39c1-87db-42a9-add1-fc676aee389d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:22.808558 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e39c1-87db-42a9-add1-fc676aee389d admin admin] PATH_INFO: `/v3/users/35f8a86ab3c241d98def02d2b2a6db6c` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:22.809748 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e39c1-87db-42a9-add1-fc676aee389d admin admin] RBAC: Authorizing `identity:delete_user(user_id=35f8a86ab3c241d98def02d2b2a6db6c)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:22.811764 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e39c1-87db-42a9-add1-fc676aee389d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:22.975096 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-674e39c1-87db-42a9-add1-fc676aee389d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35f8a86ab3c241d98def02d2b2a6db6c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:23.034668 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-674e39c1-87db-42a9-add1-fc676aee389d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35f8a86ab3c241d98def02d2b2a6db6c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:23.035852 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-674e39c1-87db-42a9-add1-fc676aee389d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35f8a86ab3c241d98def02d2b2a6db6c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:23.043332 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-674e39c1-87db-42a9-add1-fc676aee389d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35f8a86ab3c241d98def02d2b2a6db6c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:23.044564 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-674e39c1-87db-42a9-add1-fc676aee389d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35f8a86ab3c241d98def02d2b2a6db6c'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:23.061844 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1358/2720] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:43:22 2022] DELETE /identity/v3/users/35f8a86ab3c241d98def02d2b2a6db6c => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:43:23.511078 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8914616-52b4-4b95-80eb-eb692fc38918 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:23.512834 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d8914616-52b4-4b95-80eb-eb692fc38918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:23.526592 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8914616-52b4-4b95-80eb-eb692fc38918 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:23.593314 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fafb77-2784-42e7-800a-4e8f58c82ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:23.594192 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fafb77-2784-42e7-800a-4e8f58c82ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:23.594484 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fafb77-2784-42e7-800a-4e8f58c82ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:23.594760 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fafb77-2784-42e7-800a-4e8f58c82ccc admin admin] PATH_INFO: `/v3/projects/e0fffadff6764fabbcba0ff1fa6a0383` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:23.596059 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fafb77-2784-42e7-800a-4e8f58c82ccc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0fffadff6764fabbcba0ff1fa6a0383)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:23.597981 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fafb77-2784-42e7-800a-4e8f58c82ccc admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:23.709891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-73fafb77-2784-42e7-800a-4e8f58c82ccc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0fffadff6764fabbcba0ff1fa6a0383', 'request_id': 'req-73fafb77-2784-42e7-800a-4e8f58c82ccc'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:23.771558 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-73fafb77-2784-42e7-800a-4e8f58c82ccc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0fffadff6764fabbcba0ff1fa6a0383', 'request_id': 'req-73fafb77-2784-42e7-800a-4e8f58c82ccc'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:23.777595 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-73fafb77-2784-42e7-800a-4e8f58c82ccc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0fffadff6764fabbcba0ff1fa6a0383', 'request_id': 'req-73fafb77-2784-42e7-800a-4e8f58c82ccc'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:23.788666 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-73fafb77-2784-42e7-800a-4e8f58c82ccc admin admin] The token cache is being invalidate because project e0fffadff6764fabbcba0ff1fa6a0383 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:43:23.789335 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-73fafb77-2784-42e7-800a-4e8f58c82ccc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:23.789694 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-73fafb77-2784-42e7-800a-4e8f58c82ccc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:23.791375 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1363/2721] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:43:23 2022] DELETE /identity/v3/projects/e0fffadff6764fabbcba0ff1fa6a0383 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:43:23.795549 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e39c1-87db-42a9-add1-fc676aee389d admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:23.796955 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-674e39c1-87db-42a9-add1-fc676aee389d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:23.810525 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e39c1-87db-42a9-add1-fc676aee389d admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:23.850087 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:43:23.868630 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70fd22-5947-43dd-941a-db7e0d046d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:23.869577 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70fd22-5947-43dd-941a-db7e0d046d12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:23.869861 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70fd22-5947-43dd-941a-db7e0d046d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:23.870138 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70fd22-5947-43dd-941a-db7e0d046d12 admin admin] PATH_INFO: `/v3/users/80262ec36b5447d5baf4d6ddaa21a525` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:23.871363 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70fd22-5947-43dd-941a-db7e0d046d12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80262ec36b5447d5baf4d6ddaa21a525)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:23.873386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70fd22-5947-43dd-941a-db7e0d046d12 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:24.090298 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bc70fd22-5947-43dd-941a-db7e0d046d12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80262ec36b5447d5baf4d6ddaa21a525'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:24.160165 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bc70fd22-5947-43dd-941a-db7e0d046d12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80262ec36b5447d5baf4d6ddaa21a525'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:24.160802 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bc70fd22-5947-43dd-941a-db7e0d046d12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80262ec36b5447d5baf4d6ddaa21a525'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:24.168861 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bc70fd22-5947-43dd-941a-db7e0d046d12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80262ec36b5447d5baf4d6ddaa21a525'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:24.169594 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bc70fd22-5947-43dd-941a-db7e0d046d12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80262ec36b5447d5baf4d6ddaa21a525'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:24.187646 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1359/2722] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:43:23 2022] DELETE /identity/v3/users/80262ec36b5447d5baf4d6ddaa21a525 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:43:24.579234 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fafb77-2784-42e7-800a-4e8f58c82ccc admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:24.581276 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-73fafb77-2784-42e7-800a-4e8f58c82ccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:24.595627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fafb77-2784-42e7-800a-4e8f58c82ccc admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:24.653477 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d02b6-b68a-4d3c-b9a8-f06dc30e3ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:24.656459 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d02b6-b68a-4d3c-b9a8-f06dc30e3ffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:24.656459 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d02b6-b68a-4d3c-b9a8-f06dc30e3ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:24.656459 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d02b6-b68a-4d3c-b9a8-f06dc30e3ffd admin admin] PATH_INFO: `/v3/projects/cee3ad8d60154e0bab4a0c16c63e8e83` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:24.656459 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d02b6-b68a-4d3c-b9a8-f06dc30e3ffd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cee3ad8d60154e0bab4a0c16c63e8e83)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:24.658099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d02b6-b68a-4d3c-b9a8-f06dc30e3ffd admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:24.680114 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:43:24.749243 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-6f9d02b6-b68a-4d3c-b9a8-f06dc30e3ffd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cee3ad8d60154e0bab4a0c16c63e8e83', 'request_id': 'req-6f9d02b6-b68a-4d3c-b9a8-f06dc30e3ffd'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:24.840305 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-6f9d02b6-b68a-4d3c-b9a8-f06dc30e3ffd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cee3ad8d60154e0bab4a0c16c63e8e83', 'request_id': 'req-6f9d02b6-b68a-4d3c-b9a8-f06dc30e3ffd'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:24.846397 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-6f9d02b6-b68a-4d3c-b9a8-f06dc30e3ffd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cee3ad8d60154e0bab4a0c16c63e8e83', 'request_id': 'req-6f9d02b6-b68a-4d3c-b9a8-f06dc30e3ffd'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:24.858274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-6f9d02b6-b68a-4d3c-b9a8-f06dc30e3ffd admin admin] The token cache is being invalidate because project cee3ad8d60154e0bab4a0c16c63e8e83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:43:24.858569 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-6f9d02b6-b68a-4d3c-b9a8-f06dc30e3ffd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:24.859250 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-6f9d02b6-b68a-4d3c-b9a8-f06dc30e3ffd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:24.860802 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1364/2723] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:43:24 2022] DELETE /identity/v3/projects/cee3ad8d60154e0bab4a0c16c63e8e83 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:43:24.866662 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4b349-14b8-4e01-99c5-64ee6c328f43 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:24.867171 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4b349-14b8-4e01-99c5-64ee6c328f43 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:24.867269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4b349-14b8-4e01-99c5-64ee6c328f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:24.886180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-b9d4b349-14b8-4e01-99c5-64ee6c328f43 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:24.887449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b9d4b349-14b8-4e01-99c5-64ee6c328f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:24.947865 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1360/2724] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:43:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:43:24.956954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494c509-d6d7-4558-ada0-24f9c8f8c7aa None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:24.957243 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494c509-d6d7-4558-ada0-24f9c8f8c7aa None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:24.957558 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5494c509-d6d7-4558-ada0-24f9c8f8c7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:24.976356 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-5494c509-d6d7-4558-ada0-24f9c8f8c7aa None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:24.977565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5494c509-d6d7-4558-ada0-24f9c8f8c7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:24.985556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1365/2725] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:43:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:24.989938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4b349-14b8-4e01-99c5-64ee6c328f43 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:25.000364 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4b349-14b8-4e01-99c5-64ee6c328f43 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:25.005087 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4994b6c-c624-4051-bf1f-c3026bcde109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:25.005803 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4994b6c-c624-4051-bf1f-c3026bcde109 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:25.006615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4994b6c-c624-4051-bf1f-c3026bcde109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:25.006898 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4994b6c-c624-4051-bf1f-c3026bcde109 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:25.007634 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4994b6c-c624-4051-bf1f-c3026bcde109 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:43:25.007941 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4994b6c-c624-4051-bf1f-c3026bcde109 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:25.009546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4994b6c-c624-4051-bf1f-c3026bcde109 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:25.021398 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1361/2726] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:43:25.025736 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494c509-d6d7-4558-ada0-24f9c8f8c7aa None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:25.035232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5494c509-d6d7-4558-ada0-24f9c8f8c7aa None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:25.040007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18d109-ae35-4fdd-a902-99dcd5f34283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:25.040670 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18d109-ae35-4fdd-a902-99dcd5f34283 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:25.040954 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18d109-ae35-4fdd-a902-99dcd5f34283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:25.041229 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18d109-ae35-4fdd-a902-99dcd5f34283 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:25.041936 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e18d109-ae35-4fdd-a902-99dcd5f34283 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:25.043887 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e18d109-ae35-4fdd-a902-99dcd5f34283 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:25.144204 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1366/2727] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:43:25 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:43:25.149872 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4994b6c-c624-4051-bf1f-c3026bcde109 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:25.159980 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4994b6c-c624-4051-bf1f-c3026bcde109 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:25.204706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:43:25.217418 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd333b04-fa52-43b5-bc18-f42c140a27e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:25.218084 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd333b04-fa52-43b5-bc18-f42c140a27e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:25.218364 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd333b04-fa52-43b5-bc18-f42c140a27e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:25.218637 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd333b04-fa52-43b5-bc18-f42c140a27e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:25.219362 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd333b04-fa52-43b5-bc18-f42c140a27e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:25.221278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd333b04-fa52-43b5-bc18-f42c140a27e5 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:25.326509 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1362/2728] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:43:25 2022] POST /identity/v3/users => generated 287 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:43:25.332962 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18d109-ae35-4fdd-a902-99dcd5f34283 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:25.344007 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18d109-ae35-4fdd-a902-99dcd5f34283 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:25.348609 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832db587-75e9-49ef-9450-45566dad5c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:25.353478 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832db587-75e9-49ef-9450-45566dad5c55 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:25.353478 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832db587-75e9-49ef-9450-45566dad5c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:25.353478 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832db587-75e9-49ef-9450-45566dad5c55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:25.353478 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832db587-75e9-49ef-9450-45566dad5c55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:25.353478 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832db587-75e9-49ef-9450-45566dad5c55 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:25.363138 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1367/2729] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:43:25.367778 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd333b04-fa52-43b5-bc18-f42c140a27e5 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:25.378574 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd333b04-fa52-43b5-bc18-f42c140a27e5 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:25.383793 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7465107d-e171-4cf3-9ccf-b4338ff4d026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:25.386892 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465107d-e171-4cf3-9ccf-b4338ff4d026 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:25.386892 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465107d-e171-4cf3-9ccf-b4338ff4d026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:25.386892 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465107d-e171-4cf3-9ccf-b4338ff4d026 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:25.386892 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7465107d-e171-4cf3-9ccf-b4338ff4d026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:25.387380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7465107d-e171-4cf3-9ccf-b4338ff4d026 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:25.398609 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1363/2730] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:43:25.402751 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832db587-75e9-49ef-9450-45566dad5c55 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:25.412796 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832db587-75e9-49ef-9450-45566dad5c55 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:25.417400 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415e65c-b70a-4e99-8b66-d48a1df09a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:25.418286 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415e65c-b70a-4e99-8b66-d48a1df09a6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:25.418551 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415e65c-b70a-4e99-8b66-d48a1df09a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:25.418829 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415e65c-b70a-4e99-8b66-d48a1df09a6d admin admin] PATH_INFO: `/v3/projects/ca4cb6938b5e4a43bc37f2dc498838c3/users/900ea7e299e34a2d866bef87773b2a8e/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:25.435291 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b415e65c-b70a-4e99-8b66-d48a1df09a6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca4cb6938b5e4a43bc37f2dc498838c3, user_id=900ea7e299e34a2d866bef87773b2a8e, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:25.438000 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b415e65c-b70a-4e99-8b66-d48a1df09a6d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:25.509444 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1368/2731] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:43:25 2022] PUT /identity/v3/projects/ca4cb6938b5e4a43bc37f2dc498838c3/users/900ea7e299e34a2d866bef87773b2a8e/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:43:25.520196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9596b-84eb-44a4-aec1-dabdf62d1147 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:25.520486 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9596b-84eb-44a4-aec1-dabdf62d1147 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:25.520762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9596b-84eb-44a4-aec1-dabdf62d1147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:25.547080 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-d3f9596b-84eb-44a4-aec1-dabdf62d1147 None None] MFA Rules not processed for user `900ea7e299e34a2d866bef87773b2a8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:25.550117 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d3f9596b-84eb-44a4-aec1-dabdf62d1147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:25.613238 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1364/2732] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:43:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:25.624054 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415e65c-b70a-4e99-8b66-d48a1df09a6d admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:25.625744 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b415e65c-b70a-4e99-8b66-d48a1df09a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:25.685410 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415e65c-b70a-4e99-8b66-d48a1df09a6d admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:25.688976 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1a57e-7619-4712-958e-8b28bf46801f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:25.689620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1a57e-7619-4712-958e-8b28bf46801f service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:25.689900 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1a57e-7619-4712-958e-8b28bf46801f service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:25.690177 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1a57e-7619-4712-958e-8b28bf46801f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:25.699852 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1a57e-7619-4712-958e-8b28bf46801f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:25.701308 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1a57e-7619-4712-958e-8b28bf46801f service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:25.710505 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1369/2733] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:43:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:43:27.346874 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2512c2f-e6c0-470e-ab39-af10ad12de9d None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:27.351309 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2512c2f-e6c0-470e-ab39-af10ad12de9d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:27.351309 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2512c2f-e6c0-470e-ab39-af10ad12de9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:27.366345 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-f2512c2f-e6c0-470e-ab39-af10ad12de9d None None] MFA Rules not processed for user `900ea7e299e34a2d866bef87773b2a8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:27.367566 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f2512c2f-e6c0-470e-ab39-af10ad12de9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:27.374373 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1365/2734] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:43:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:29.252032 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1a57e-7619-4712-958e-8b28bf46801f service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:29.253388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-c8d1a57e-7619-4712-958e-8b28bf46801f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:29.301139 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:43:29.321378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1a57e-7619-4712-958e-8b28bf46801f service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:29.326367 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3b014-5803-42c2-980a-04d6f62a6ab1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f2d0aceef4451a83ee89fac5130a0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:29.327032 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3b014-5803-42c2-980a-04d6f62a6ab1 service placement] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:29.327313 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3b014-5803-42c2-980a-04d6f62a6ab1 service placement] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:29.327583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a3b014-5803-42c2-980a-04d6f62a6ab1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:29.329332 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-96a3b014-5803-42c2-980a-04d6f62a6ab1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:29.341615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a3b014-5803-42c2-980a-04d6f62a6ab1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:29.343222 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a3b014-5803-42c2-980a-04d6f62a6ab1 service placement] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:29.353103 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1370/2735] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:43:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:43:37.361052 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=48,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:43:39.399724 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=11 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:43:46.222405 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722a156-4382-4e9b-a4c1-810eecb8ba3b None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:46.222925 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722a156-4382-4e9b-a4c1-810eecb8ba3b None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:46.223041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722a156-4382-4e9b-a4c1-810eecb8ba3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:46.240652 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-0722a156-4382-4e9b-a4c1-810eecb8ba3b None None] MFA Rules not processed for user `07c9688ca035484ba5341859574c2799`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:46.241910 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-0722a156-4382-4e9b-a4c1-810eecb8ba3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:46.295489 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1366/2736] 10.222.0.24 () {58 vars in 1009 bytes} [Fri May 27 03:43:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:46.420356 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3b014-5803-42c2-980a-04d6f62a6ab1 service placement] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:46.421776 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-96a3b014-5803-42c2-980a-04d6f62a6ab1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:46.490280 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a3b014-5803-42c2-980a-04d6f62a6ab1 service placement] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:46.495146 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b8f95-7020-4697-93e0-85ac24dccc7c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d559cd7127490da84185f20684929d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:46.495791 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b8f95-7020-4697-93e0-85ac24dccc7c service swift] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:46.496073 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b8f95-7020-4697-93e0-85ac24dccc7c service swift] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:46.496349 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b8f95-7020-4697-93e0-85ac24dccc7c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:46.504814 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b8f95-7020-4697-93e0-85ac24dccc7c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:46.506378 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b8f95-7020-4697-93e0-85ac24dccc7c service swift] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:46.515195 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1371/2737] 10.222.0.24 () {60 vars in 1384 bytes} [Fri May 27 03:43:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:43:47.286170 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771317d-93c6-4f7b-bb47-d306f7bd0d56 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:47.286673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771317d-93c6-4f7b-bb47-d306f7bd0d56 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:47.286803 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771317d-93c6-4f7b-bb47-d306f7bd0d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:47.304778 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-6771317d-93c6-4f7b-bb47-d306f7bd0d56 None None] MFA Rules not processed for user `07c9688ca035484ba5341859574c2799`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:47.305967 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-6771317d-93c6-4f7b-bb47-d306f7bd0d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:47.313042 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1367/2738] 10.222.0.24 () {58 vars in 967 bytes} [Fri May 27 03:43:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:47.324825 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b8f95-7020-4697-93e0-85ac24dccc7c service swift] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:47.336585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b8f95-7020-4697-93e0-85ac24dccc7c service swift] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:47.343433 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110befc8-def7-4c15-8742-0a30bc58f966 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d559cd7127490da84185f20684929d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:47.344077 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110befc8-def7-4c15-8742-0a30bc58f966 service swift] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:47.344362 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110befc8-def7-4c15-8742-0a30bc58f966 service swift] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:47.344639 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110befc8-def7-4c15-8742-0a30bc58f966 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:47.352947 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110befc8-def7-4c15-8742-0a30bc58f966 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:47.354523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110befc8-def7-4c15-8742-0a30bc58f966 service swift] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:47.363274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1372/2739] 10.222.0.24 () {60 vars in 1384 bytes} [Fri May 27 03:43:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:43:47.964417 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbdd87-99c8-404d-a3a7-e99e87a1e46f None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:47.964706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbdd87-99c8-404d-a3a7-e99e87a1e46f None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:47.964981 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fbdd87-99c8-404d-a3a7-e99e87a1e46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:47.982449 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-37fbdd87-99c8-404d-a3a7-e99e87a1e46f None None] MFA Rules not processed for user `07c9688ca035484ba5341859574c2799`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:47.983793 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-37fbdd87-99c8-404d-a3a7-e99e87a1e46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:47.991070 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1368/2740] 10.222.0.24 () {58 vars in 1009 bytes} [Fri May 27 03:43:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:48.003482 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110befc8-def7-4c15-8742-0a30bc58f966 service swift] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:48.013859 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110befc8-def7-4c15-8742-0a30bc58f966 service swift] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:48.019499 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a32dd-24b0-48e6-a419-0a02cdb8f67f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d559cd7127490da84185f20684929d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:48.020162 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a32dd-24b0-48e6-a419-0a02cdb8f67f service swift] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:48.020461 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a32dd-24b0-48e6-a419-0a02cdb8f67f service swift] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:48.020734 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a32dd-24b0-48e6-a419-0a02cdb8f67f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:48.028826 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a32dd-24b0-48e6-a419-0a02cdb8f67f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:48.030428 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a32dd-24b0-48e6-a419-0a02cdb8f67f service swift] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:48.038446 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1373/2741] 10.222.0.24 () {60 vars in 1384 bytes} [Fri May 27 03:43:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:43:48.668783 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71545cff-e123-4b8c-a2bb-aad4d6991491 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:48.669251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71545cff-e123-4b8c-a2bb-aad4d6991491 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:48.669367 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71545cff-e123-4b8c-a2bb-aad4d6991491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:48.688638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-71545cff-e123-4b8c-a2bb-aad4d6991491 None None] MFA Rules not processed for user `07c9688ca035484ba5341859574c2799`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:48.689819 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-71545cff-e123-4b8c-a2bb-aad4d6991491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:48.696922 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1369/2742] 10.222.0.24 () {58 vars in 967 bytes} [Fri May 27 03:43:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:48.708976 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a32dd-24b0-48e6-a419-0a02cdb8f67f service swift] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:48.719900 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a32dd-24b0-48e6-a419-0a02cdb8f67f service swift] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:48.724910 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e18b0-fb9c-4bf7-bc8d-635413a4cdaa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21d559cd7127490da84185f20684929d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:48.725540 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e18b0-fb9c-4bf7-bc8d-635413a4cdaa service swift] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:48.725824 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e18b0-fb9c-4bf7-bc8d-635413a4cdaa service swift] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:48.726100 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e18b0-fb9c-4bf7-bc8d-635413a4cdaa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:48.734623 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177e18b0-fb9c-4bf7-bc8d-635413a4cdaa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:48.736229 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177e18b0-fb9c-4bf7-bc8d-635413a4cdaa service swift] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:48.744242 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1374/2743] 10.222.0.24 () {60 vars in 1384 bytes} [Fri May 27 03:43:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:43:53.590276 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71545cff-e123-4b8c-a2bb-aad4d6991491 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:53.656707 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71545cff-e123-4b8c-a2bb-aad4d6991491 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:53.661349 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829cb01-0438-42d1-8c0f-fa6cda100f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:53.662227 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829cb01-0438-42d1-8c0f-fa6cda100f21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:53.662527 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829cb01-0438-42d1-8c0f-fa6cda100f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:53.662792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829cb01-0438-42d1-8c0f-fa6cda100f21 admin admin] PATH_INFO: `/v3/users/900ea7e299e34a2d866bef87773b2a8e` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:53.664011 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829cb01-0438-42d1-8c0f-fa6cda100f21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=900ea7e299e34a2d866bef87773b2a8e)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:53.665974 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829cb01-0438-42d1-8c0f-fa6cda100f21 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:53.876072 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1829cb01-0438-42d1-8c0f-fa6cda100f21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '900ea7e299e34a2d866bef87773b2a8e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:53.993319 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1829cb01-0438-42d1-8c0f-fa6cda100f21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '900ea7e299e34a2d866bef87773b2a8e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:54.011017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1829cb01-0438-42d1-8c0f-fa6cda100f21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '900ea7e299e34a2d866bef87773b2a8e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:54.011017 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1829cb01-0438-42d1-8c0f-fa6cda100f21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '900ea7e299e34a2d866bef87773b2a8e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:54.012197 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-1829cb01-0438-42d1-8c0f-fa6cda100f21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '900ea7e299e34a2d866bef87773b2a8e'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:54.034997 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1370/2744] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:43:53 2022] DELETE /identity/v3/users/900ea7e299e34a2d866bef87773b2a8e => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:43:54.040538 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a2ffd-6420-452e-91a1-f2efc59f97c8 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:54.041054 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a2ffd-6420-452e-91a1-f2efc59f97c8 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:54.041156 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a2ffd-6420-452e-91a1-f2efc59f97c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:54.059588 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-da9a2ffd-6420-452e-91a1-f2efc59f97c8 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:54.060924 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-da9a2ffd-6420-452e-91a1-f2efc59f97c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:54.142362 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1375/2745] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:43:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:54.154768 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829cb01-0438-42d1-8c0f-fa6cda100f21 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:54.156764 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1829cb01-0438-42d1-8c0f-fa6cda100f21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:54.174607 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:43:54.224173 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829cb01-0438-42d1-8c0f-fa6cda100f21 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:54.227813 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f905d59-4e50-47ca-b200-37b07cf22404 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:54.228448 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f905d59-4e50-47ca-b200-37b07cf22404 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:54.228730 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f905d59-4e50-47ca-b200-37b07cf22404 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:54.229003 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f905d59-4e50-47ca-b200-37b07cf22404 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:54.239269 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f905d59-4e50-47ca-b200-37b07cf22404 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:54.240732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f905d59-4e50-47ca-b200-37b07cf22404 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:54.250763 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1371/2746] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:43:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:43:54.556939 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a2ffd-6420-452e-91a1-f2efc59f97c8 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:54.558439 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-da9a2ffd-6420-452e-91a1-f2efc59f97c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:54.571254 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a2ffd-6420-452e-91a1-f2efc59f97c8 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:54.576597 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00bd3bf-1ab3-4fec-99ca-786d9883ff10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:54.578750 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00bd3bf-1ab3-4fec-99ca-786d9883ff10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:54.579147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00bd3bf-1ab3-4fec-99ca-786d9883ff10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:54.579489 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00bd3bf-1ab3-4fec-99ca-786d9883ff10 admin admin] PATH_INFO: `/v3/projects/ca4cb6938b5e4a43bc37f2dc498838c3` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:54.580882 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00bd3bf-1ab3-4fec-99ca-786d9883ff10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca4cb6938b5e4a43bc37f2dc498838c3)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:54.584134 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00bd3bf-1ab3-4fec-99ca-786d9883ff10 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:54.734891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:43:54.737585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d00bd3bf-1ab3-4fec-99ca-786d9883ff10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca4cb6938b5e4a43bc37f2dc498838c3', 'request_id': 'req-d00bd3bf-1ab3-4fec-99ca-786d9883ff10'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:54.902315 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d00bd3bf-1ab3-4fec-99ca-786d9883ff10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca4cb6938b5e4a43bc37f2dc498838c3', 'request_id': 'req-d00bd3bf-1ab3-4fec-99ca-786d9883ff10'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:54.908200 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d00bd3bf-1ab3-4fec-99ca-786d9883ff10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca4cb6938b5e4a43bc37f2dc498838c3', 'request_id': 'req-d00bd3bf-1ab3-4fec-99ca-786d9883ff10'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:54.919441 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d00bd3bf-1ab3-4fec-99ca-786d9883ff10 admin admin] The token cache is being invalidate because project ca4cb6938b5e4a43bc37f2dc498838c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:43:54.919734 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d00bd3bf-1ab3-4fec-99ca-786d9883ff10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:54.920374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-d00bd3bf-1ab3-4fec-99ca-786d9883ff10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:43:54.921961 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1376/2747] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:43:54 2022] DELETE /identity/v3/projects/ca4cb6938b5e4a43bc37f2dc498838c3 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:43:54.926720 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a125a8-69bd-4ac4-9b05-83263bb803c5 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:54.927228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a125a8-69bd-4ac4-9b05-83263bb803c5 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:54.927335 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a125a8-69bd-4ac4-9b05-83263bb803c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:54.945759 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-85a125a8-69bd-4ac4-9b05-83263bb803c5 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:54.946999 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-85a125a8-69bd-4ac4-9b05-83263bb803c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:55.004755 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1372/2748] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:43:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:55.012389 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d675d7d6-66f3-4c37-9946-60e7c232d8f4 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:55.012809 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d675d7d6-66f3-4c37-9946-60e7c232d8f4 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:55.013085 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d675d7d6-66f3-4c37-9946-60e7c232d8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:55.031944 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-d675d7d6-66f3-4c37-9946-60e7c232d8f4 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:55.033225 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d675d7d6-66f3-4c37-9946-60e7c232d8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:55.041096 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1377/2749] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:55.045233 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a125a8-69bd-4ac4-9b05-83263bb803c5 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:55.055460 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a125a8-69bd-4ac4-9b05-83263bb803c5 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:55.060000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef401d2-0267-4aa0-8427-2b510d92bafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:55.060650 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef401d2-0267-4aa0-8427-2b510d92bafd admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:55.060937 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef401d2-0267-4aa0-8427-2b510d92bafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:55.061216 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef401d2-0267-4aa0-8427-2b510d92bafd admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:55.062554 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef401d2-0267-4aa0-8427-2b510d92bafd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:43:55.062919 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef401d2-0267-4aa0-8427-2b510d92bafd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:55.064526 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef401d2-0267-4aa0-8427-2b510d92bafd admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:55.076768 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1373/2750] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:43:55.080117 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d675d7d6-66f3-4c37-9946-60e7c232d8f4 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:55.089722 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d675d7d6-66f3-4c37-9946-60e7c232d8f4 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:55.094486 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f6982-c251-495f-a00f-86c60c5a3614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:55.095171 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f6982-c251-495f-a00f-86c60c5a3614 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:55.095455 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f6982-c251-495f-a00f-86c60c5a3614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:55.095729 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24f6982-c251-495f-a00f-86c60c5a3614 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:55.096414 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f6982-c251-495f-a00f-86c60c5a3614 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:55.098316 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24f6982-c251-495f-a00f-86c60c5a3614 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:55.183263 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1378/2751] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:43:55 2022] POST /identity/v3/projects => generated 389 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:43:55.188006 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef401d2-0267-4aa0-8427-2b510d92bafd admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:55.197994 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef401d2-0267-4aa0-8427-2b510d92bafd admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:55.252732 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a35a04-3253-43e3-aaf1-4e7b9deba182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:55.253396 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a35a04-3253-43e3-aaf1-4e7b9deba182 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:55.253677 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a35a04-3253-43e3-aaf1-4e7b9deba182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:55.253949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a35a04-3253-43e3-aaf1-4e7b9deba182 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:55.254623 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a35a04-3253-43e3-aaf1-4e7b9deba182 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:55.256631 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a35a04-3253-43e3-aaf1-4e7b9deba182 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:55.324886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1374/2752] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:43:55 2022] POST /identity/v3/users => generated 295 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:43:55.328809 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f6982-c251-495f-a00f-86c60c5a3614 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:55.339246 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24f6982-c251-495f-a00f-86c60c5a3614 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:55.343803 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670ca14-b828-4010-84f6-4b0e8c876a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:55.344456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670ca14-b828-4010-84f6-4b0e8c876a7f admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:55.344740 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670ca14-b828-4010-84f6-4b0e8c876a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:55.345018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d670ca14-b828-4010-84f6-4b0e8c876a7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:55.345741 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d670ca14-b828-4010-84f6-4b0e8c876a7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:55.347370 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d670ca14-b828-4010-84f6-4b0e8c876a7f admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:55.357597 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1379/2753] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:43:55.362137 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a35a04-3253-43e3-aaf1-4e7b9deba182 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:55.372036 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a35a04-3253-43e3-aaf1-4e7b9deba182 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:55.377292 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298088df-2bbb-47a0-896f-9c7f6ee90de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:55.377942 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298088df-2bbb-47a0-896f-9c7f6ee90de7 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:55.378242 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298088df-2bbb-47a0-896f-9c7f6ee90de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:55.378504 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298088df-2bbb-47a0-896f-9c7f6ee90de7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:55.379266 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298088df-2bbb-47a0-896f-9c7f6ee90de7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:55.380869 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298088df-2bbb-47a0-896f-9c7f6ee90de7 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:55.390642 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1375/2754] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:43:55.394635 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670ca14-b828-4010-84f6-4b0e8c876a7f admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:55.404396 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d670ca14-b828-4010-84f6-4b0e8c876a7f admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:55.408974 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e2899-cce7-4b6a-806f-a6f539c12b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:55.409842 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e2899-cce7-4b6a-806f-a6f539c12b71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:55.410126 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e2899-cce7-4b6a-806f-a6f539c12b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:55.410405 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e2899-cce7-4b6a-806f-a6f539c12b71 admin admin] PATH_INFO: `/v3/projects/57e1b7520fd345dda3a76ca7ddeea846/users/4fb8863014fa4a2cbdbf91ec8d1f422d/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:55.425063 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e2899-cce7-4b6a-806f-a6f539c12b71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57e1b7520fd345dda3a76ca7ddeea846, user_id=4fb8863014fa4a2cbdbf91ec8d1f422d, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:55.427747 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524e2899-cce7-4b6a-806f-a6f539c12b71 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:55.501767 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1380/2755] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:43:55 2022] PUT /identity/v3/projects/57e1b7520fd345dda3a76ca7ddeea846/users/4fb8863014fa4a2cbdbf91ec8d1f422d/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:43:55.511858 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124c4a3-d979-40ab-a57d-e8fe8def7910 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:55.512097 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124c4a3-d979-40ab-a57d-e8fe8def7910 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:55.512373 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124c4a3-d979-40ab-a57d-e8fe8def7910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:55.536718 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:43:55.539340 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-1124c4a3-d979-40ab-a57d-e8fe8def7910 None None] MFA Rules not processed for user `4fb8863014fa4a2cbdbf91ec8d1f422d`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:43:55.540547 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1124c4a3-d979-40ab-a57d-e8fe8def7910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:55.605616 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1376/2756] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:43:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:43:55.616783 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e2899-cce7-4b6a-806f-a6f539c12b71 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:43:55.618471 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-524e2899-cce7-4b6a-806f-a6f539c12b71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:43:55.679630 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e2899-cce7-4b6a-806f-a6f539c12b71 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:43:55.683328 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c48731f-13a1-48e2-be81-8885c3c4aaf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:43:55.683978 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48731f-13a1-48e2-be81-8885c3c4aaf2 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:43:55.684263 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48731f-13a1-48e2-be81-8885c3c4aaf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:43:55.684541 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48731f-13a1-48e2-be81-8885c3c4aaf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:43:55.693910 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c48731f-13a1-48e2-be81-8885c3c4aaf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:43:55.695398 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c48731f-13a1-48e2-be81-8885c3c4aaf2 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:43:55.704343 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1381/2757] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:43:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:44:05.606782 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=26 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:44:05.700222 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:DELETE=4,keystone:SELECT=60,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:44:16.523246 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124c4a3-d979-40ab-a57d-e8fe8def7910 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:44:16.582422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124c4a3-d979-40ab-a57d-e8fe8def7910 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:44:16.587168 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc96429-220e-4761-9968-1473e4a90c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:44:16.588025 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc96429-220e-4761-9968-1473e4a90c34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:44:16.588302 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc96429-220e-4761-9968-1473e4a90c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:44:16.588573 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc96429-220e-4761-9968-1473e4a90c34 admin admin] PATH_INFO: `/v3/users/4fb8863014fa4a2cbdbf91ec8d1f422d` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:44:16.589766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc96429-220e-4761-9968-1473e4a90c34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fb8863014fa4a2cbdbf91ec8d1f422d)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:44:16.591722 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc96429-220e-4761-9968-1473e4a90c34 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:44:16.738750 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-2cc96429-220e-4761-9968-1473e4a90c34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fb8863014fa4a2cbdbf91ec8d1f422d'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:44:16.804583 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-2cc96429-220e-4761-9968-1473e4a90c34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fb8863014fa4a2cbdbf91ec8d1f422d'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:44:16.805204 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-2cc96429-220e-4761-9968-1473e4a90c34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fb8863014fa4a2cbdbf91ec8d1f422d'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:44:16.812677 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-2cc96429-220e-4761-9968-1473e4a90c34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fb8863014fa4a2cbdbf91ec8d1f422d'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:44:16.813358 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-2cc96429-220e-4761-9968-1473e4a90c34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fb8863014fa4a2cbdbf91ec8d1f422d'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:44:16.831193 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1377/2758] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:44:16 2022] DELETE /identity/v3/users/4fb8863014fa4a2cbdbf91ec8d1f422d => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:44:16.835969 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1067a-c201-4301-a9e9-9625bb1dd46f None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:44:16.836426 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1067a-c201-4301-a9e9-9625bb1dd46f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:44:16.836522 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1067a-c201-4301-a9e9-9625bb1dd46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:44:16.854352 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-60b1067a-c201-4301-a9e9-9625bb1dd46f None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:44:16.855624 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-60b1067a-c201-4301-a9e9-9625bb1dd46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:44:16.915721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1382/2759] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:44:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:44:16.925351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc96429-220e-4761-9968-1473e4a90c34 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:44:16.926818 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-2cc96429-220e-4761-9968-1473e4a90c34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:44:16.983216 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc96429-220e-4761-9968-1473e4a90c34 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:44:16.986916 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7890b8a1-e580-4086-8724-0530fad0f054 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:44:16.987559 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890b8a1-e580-4086-8724-0530fad0f054 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:44:16.987838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890b8a1-e580-4086-8724-0530fad0f054 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:44:16.988113 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7890b8a1-e580-4086-8724-0530fad0f054 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:44:16.997215 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7890b8a1-e580-4086-8724-0530fad0f054 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:44:16.998649 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7890b8a1-e580-4086-8724-0530fad0f054 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:44:17.008144 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1378/2760] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:44:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:44:17.424512 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1067a-c201-4301-a9e9-9625bb1dd46f None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:44:17.426050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-60b1067a-c201-4301-a9e9-9625bb1dd46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:44:17.442897 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1067a-c201-4301-a9e9-9625bb1dd46f None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:44:17.451018 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bcfdb0-5eac-4438-b479-a0d6d0b8cef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:44:17.451894 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcfdb0-5eac-4438-b479-a0d6d0b8cef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:44:17.452178 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcfdb0-5eac-4438-b479-a0d6d0b8cef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:44:17.452473 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcfdb0-5eac-4438-b479-a0d6d0b8cef4 admin admin] PATH_INFO: `/v3/projects/57e1b7520fd345dda3a76ca7ddeea846` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:44:17.453765 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bcfdb0-5eac-4438-b479-a0d6d0b8cef4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57e1b7520fd345dda3a76ca7ddeea846)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:44:17.455755 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bcfdb0-5eac-4438-b479-a0d6d0b8cef4 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:44:17.576442 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-16bcfdb0-5eac-4438-b479-a0d6d0b8cef4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57e1b7520fd345dda3a76ca7ddeea846', 'request_id': 'req-16bcfdb0-5eac-4438-b479-a0d6d0b8cef4'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:44:17.644493 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-16bcfdb0-5eac-4438-b479-a0d6d0b8cef4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57e1b7520fd345dda3a76ca7ddeea846', 'request_id': 'req-16bcfdb0-5eac-4438-b479-a0d6d0b8cef4'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:44:17.650171 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-16bcfdb0-5eac-4438-b479-a0d6d0b8cef4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57e1b7520fd345dda3a76ca7ddeea846', 'request_id': 'req-16bcfdb0-5eac-4438-b479-a0d6d0b8cef4'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:44:17.661801 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-16bcfdb0-5eac-4438-b479-a0d6d0b8cef4 admin admin] The token cache is being invalidate because project 57e1b7520fd345dda3a76ca7ddeea846 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:44:17.662065 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-16bcfdb0-5eac-4438-b479-a0d6d0b8cef4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:44:17.662703 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-16bcfdb0-5eac-4438-b479-a0d6d0b8cef4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:44:17.664266 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1383/2761] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:44:17 2022] DELETE /identity/v3/projects/57e1b7520fd345dda3a76ca7ddeea846 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:44:23.626088 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c2e59-31bc-47f8-a263-1a9515a20d2d None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:44:23.626593 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c2e59-31bc-47f8-a263-1a9515a20d2d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:44:23.626723 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025c2e59-31bc-47f8-a263-1a9515a20d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:44:23.645661 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-025c2e59-31bc-47f8-a263-1a9515a20d2d None None] MFA Rules not processed for user `1e33fbb258684a00a7a8d948d92a8c01`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:44:23.646902 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-025c2e59-31bc-47f8-a263-1a9515a20d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:44:23.663087 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:44:23.700551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1379/2762] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:44:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:44:27.658730 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:44:33.764472 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=15 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:44:58.093042 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb9577-bf1f-488b-b21b-76ef857b4e95 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:44:58.093565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb9577-bf1f-488b-b21b-76ef857b4e95 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:44:58.093565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccb9577-bf1f-488b-b21b-76ef857b4e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:44:58.111825 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-7ccb9577-bf1f-488b-b21b-76ef857b4e95 None None] MFA Rules not processed for user `2e4436dc550a44269ffbd4760670f551`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:44:58.113036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-7ccb9577-bf1f-488b-b21b-76ef857b4e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:44:58.166649 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1384/2763] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:44:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:45:08.108983 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c2e59-31bc-47f8-a263-1a9515a20d2d None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:45:08.110472 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-025c2e59-31bc-47f8-a263-1a9515a20d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:45:08.159388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=25 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:45:08.168853 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025c2e59-31bc-47f8-a263-1a9515a20d2d None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:45:08.172490 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627821e-aaf1-481d-93a3-babe8e296be1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:45:08.173141 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627821e-aaf1-481d-93a3-babe8e296be1 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:45:08.173446 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627821e-aaf1-481d-93a3-babe8e296be1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:45:08.173722 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627821e-aaf1-481d-93a3-babe8e296be1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:45:08.175629 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-7627821e-aaf1-481d-93a3-babe8e296be1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:45:08.250404 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627821e-aaf1-481d-93a3-babe8e296be1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:45:08.251923 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627821e-aaf1-481d-93a3-babe8e296be1 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:45:08.265294 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1380/2764] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:45:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:45:09.443806 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb9577-bf1f-488b-b21b-76ef857b4e95 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:45:09.454849 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccb9577-bf1f-488b-b21b-76ef857b4e95 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:45:09.458320 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510d8966-d49f-4dc7-8098-729c4b88b775 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:45:09.459033 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d8966-d49f-4dc7-8098-729c4b88b775 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:45:09.459603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d8966-d49f-4dc7-8098-729c4b88b775 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:45:09.463208 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d8966-d49f-4dc7-8098-729c4b88b775 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:45:09.463208 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-510d8966-d49f-4dc7-8098-729c4b88b775 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:45:09.475377 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510d8966-d49f-4dc7-8098-729c4b88b775 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:45:09.476799 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510d8966-d49f-4dc7-8098-729c4b88b775 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:45:09.492000 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1385/2765] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:45:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:45:18.260058 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=54 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:45:19.483067 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=6 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:45:28.964057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627821e-aaf1-481d-93a3-babe8e296be1 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:45:28.975669 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-7627821e-aaf1-481d-93a3-babe8e296be1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:45:29.029531 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627821e-aaf1-481d-93a3-babe8e296be1 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:45:29.034077 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20253b36-5068-4f80-b0f7-e78fe26af89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:45:29.034972 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20253b36-5068-4f80-b0f7-e78fe26af89f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:45:29.035258 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20253b36-5068-4f80-b0f7-e78fe26af89f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:45:29.035538 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20253b36-5068-4f80-b0f7-e78fe26af89f admin admin] PATH_INFO: `/v3/users/1e33fbb258684a00a7a8d948d92a8c01` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:45:29.036721 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20253b36-5068-4f80-b0f7-e78fe26af89f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e33fbb258684a00a7a8d948d92a8c01)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:45:29.038658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20253b36-5068-4f80-b0f7-e78fe26af89f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:45:29.279688 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-20253b36-5068-4f80-b0f7-e78fe26af89f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e33fbb258684a00a7a8d948d92a8c01'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:45:29.368596 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-20253b36-5068-4f80-b0f7-e78fe26af89f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e33fbb258684a00a7a8d948d92a8c01'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:45:29.369220 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-20253b36-5068-4f80-b0f7-e78fe26af89f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e33fbb258684a00a7a8d948d92a8c01'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:45:29.376351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-20253b36-5068-4f80-b0f7-e78fe26af89f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e33fbb258684a00a7a8d948d92a8c01'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:45:29.376971 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-20253b36-5068-4f80-b0f7-e78fe26af89f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e33fbb258684a00a7a8d948d92a8c01'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:45:29.393799 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1381/2766] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:45:28 2022] DELETE /identity/v3/users/1e33fbb258684a00a7a8d948d92a8c01 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:45:29.672693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510d8966-d49f-4dc7-8098-729c4b88b775 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:45:29.683818 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510d8966-d49f-4dc7-8098-729c4b88b775 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:45:29.740654 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd603f-480a-4c07-b9d1-2d22a3075892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:45:29.741510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd603f-480a-4c07-b9d1-2d22a3075892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:45:29.741788 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd603f-480a-4c07-b9d1-2d22a3075892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:45:29.742058 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd603f-480a-4c07-b9d1-2d22a3075892 admin admin] PATH_INFO: `/v3/projects/245626825e3b480485223cd0e3d54580` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:45:29.743250 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd603f-480a-4c07-b9d1-2d22a3075892 admin admin] RBAC: Authorizing `identity:delete_project(project_id=245626825e3b480485223cd0e3d54580)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:45:29.745143 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd603f-480a-4c07-b9d1-2d22a3075892 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:45:29.851699 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-01dd603f-480a-4c07-b9d1-2d22a3075892 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '245626825e3b480485223cd0e3d54580', 'request_id': 'req-01dd603f-480a-4c07-b9d1-2d22a3075892'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:45:29.922900 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-01dd603f-480a-4c07-b9d1-2d22a3075892 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '245626825e3b480485223cd0e3d54580', 'request_id': 'req-01dd603f-480a-4c07-b9d1-2d22a3075892'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:45:29.928899 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-01dd603f-480a-4c07-b9d1-2d22a3075892 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '245626825e3b480485223cd0e3d54580', 'request_id': 'req-01dd603f-480a-4c07-b9d1-2d22a3075892'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:45:29.940584 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-01dd603f-480a-4c07-b9d1-2d22a3075892 admin admin] The token cache is being invalidate because project 245626825e3b480485223cd0e3d54580 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:45:29.940864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-01dd603f-480a-4c07-b9d1-2d22a3075892 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:45:29.941471 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-01dd603f-480a-4c07-b9d1-2d22a3075892 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:45:29.943022 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1386/2767] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:45:29 2022] DELETE /identity/v3/projects/245626825e3b480485223cd0e3d54580 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:45:39.390305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:45:39.937434 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:46:16.679159 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20253b36-5068-4f80-b0f7-e78fe26af89f admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:16.680629 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-20253b36-5068-4f80-b0f7-e78fe26af89f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:46:16.759843 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20253b36-5068-4f80-b0f7-e78fe26af89f admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:16.831235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6607026-6a18-4e61-a066-56f915b668e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:16.832482 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6607026-6a18-4e61-a066-56f915b668e9 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:16.832482 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6607026-6a18-4e61-a066-56f915b668e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:16.832705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6607026-6a18-4e61-a066-56f915b668e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:16.835736 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f6607026-6a18-4e61-a066-56f915b668e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:46:16.851233 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6607026-6a18-4e61-a066-56f915b668e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:16.852886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6607026-6a18-4e61-a066-56f915b668e9 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:16.865760 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1382/2768] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:46:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:46:17.345936 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd603f-480a-4c07-b9d1-2d22a3075892 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:17.347430 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-01dd603f-480a-4c07-b9d1-2d22a3075892 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:46:17.432362 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd603f-480a-4c07-b9d1-2d22a3075892 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:17.436164 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae012bf8-693c-40b3-aabf-a72bbbdc7590 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:17.436917 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae012bf8-693c-40b3-aabf-a72bbbdc7590 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:17.437208 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae012bf8-693c-40b3-aabf-a72bbbdc7590 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:17.437487 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae012bf8-693c-40b3-aabf-a72bbbdc7590 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:17.439444 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ae012bf8-693c-40b3-aabf-a72bbbdc7590 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:46:17.451113 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae012bf8-693c-40b3-aabf-a72bbbdc7590 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:17.453169 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae012bf8-693c-40b3-aabf-a72bbbdc7590 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:17.461741 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1387/2769] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:46:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:46:26.630436 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6607026-6a18-4e61-a066-56f915b668e9 service nova] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:26.632417 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f6607026-6a18-4e61-a066-56f915b668e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:46:26.706798 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6607026-6a18-4e61-a066-56f915b668e9 service nova] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:26.713598 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad92be8-1c21-4697-845a-7441a925fd53 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f2d0aceef4451a83ee89fac5130a0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:26.714362 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad92be8-1c21-4697-845a-7441a925fd53 service placement] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:26.714716 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad92be8-1c21-4697-845a-7441a925fd53 service placement] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:26.715116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad92be8-1c21-4697-845a-7441a925fd53 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:26.724381 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1ad92be8-1c21-4697-845a-7441a925fd53 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:46:26.736595 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad92be8-1c21-4697-845a-7441a925fd53 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:26.738347 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad92be8-1c21-4697-845a-7441a925fd53 service placement] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:26.748781 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1383/2770] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:46:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:46:26.771379 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae012bf8-693c-40b3-aabf-a72bbbdc7590 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:26.773714 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ae012bf8-693c-40b3-aabf-a72bbbdc7590 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:46:26.852107 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae012bf8-693c-40b3-aabf-a72bbbdc7590 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:26.856466 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5e134-f115-4cff-842f-eb6d817196ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:26.857331 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5e134-f115-4cff-842f-eb6d817196ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:26.857607 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5e134-f115-4cff-842f-eb6d817196ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:26.857875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5e134-f115-4cff-842f-eb6d817196ab admin admin] PATH_INFO: `/v3/users/2e4436dc550a44269ffbd4760670f551` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:26.859100 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5e134-f115-4cff-842f-eb6d817196ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e4436dc550a44269ffbd4760670f551)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:26.861010 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5e134-f115-4cff-842f-eb6d817196ab admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:27.034253 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-82d5e134-f115-4cff-842f-eb6d817196ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e4436dc550a44269ffbd4760670f551'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:27.093745 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-82d5e134-f115-4cff-842f-eb6d817196ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e4436dc550a44269ffbd4760670f551'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:27.106910 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-82d5e134-f115-4cff-842f-eb6d817196ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e4436dc550a44269ffbd4760670f551'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:27.107506 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-82d5e134-f115-4cff-842f-eb6d817196ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e4436dc550a44269ffbd4760670f551'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:27.114396 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-82d5e134-f115-4cff-842f-eb6d817196ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e4436dc550a44269ffbd4760670f551'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:27.120893 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1388/2771] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:46:26 2022] DELETE /identity/v3/users/2e4436dc550a44269ffbd4760670f551 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:46:27.364525 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad92be8-1c21-4697-845a-7441a925fd53 service placement] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:27.376492 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad92be8-1c21-4697-845a-7441a925fd53 service placement] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:27.386870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=100 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:46:27.433734 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f0ee1-5a42-4e1c-9b73-2dcb9d5bc66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:27.434598 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f0ee1-5a42-4e1c-9b73-2dcb9d5bc66d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:27.434924 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f0ee1-5a42-4e1c-9b73-2dcb9d5bc66d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:27.435205 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f0ee1-5a42-4e1c-9b73-2dcb9d5bc66d admin admin] PATH_INFO: `/v3/projects/37f34daf6ac94c41b23e1bc899ab671f` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:27.436397 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f0ee1-5a42-4e1c-9b73-2dcb9d5bc66d admin admin] RBAC: Authorizing `identity:delete_project(project_id=37f34daf6ac94c41b23e1bc899ab671f)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:27.438309 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f0ee1-5a42-4e1c-9b73-2dcb9d5bc66d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:27.515823 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-433f0ee1-5a42-4e1c-9b73-2dcb9d5bc66d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37f34daf6ac94c41b23e1bc899ab671f', 'request_id': 'req-433f0ee1-5a42-4e1c-9b73-2dcb9d5bc66d'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:27.567814 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-433f0ee1-5a42-4e1c-9b73-2dcb9d5bc66d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37f34daf6ac94c41b23e1bc899ab671f', 'request_id': 'req-433f0ee1-5a42-4e1c-9b73-2dcb9d5bc66d'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:27.574971 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-433f0ee1-5a42-4e1c-9b73-2dcb9d5bc66d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37f34daf6ac94c41b23e1bc899ab671f', 'request_id': 'req-433f0ee1-5a42-4e1c-9b73-2dcb9d5bc66d'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:27.586148 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-433f0ee1-5a42-4e1c-9b73-2dcb9d5bc66d admin admin] The token cache is being invalidate because project 37f34daf6ac94c41b23e1bc899ab671f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:46:27.586434 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-433f0ee1-5a42-4e1c-9b73-2dcb9d5bc66d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:27.587097 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-433f0ee1-5a42-4e1c-9b73-2dcb9d5bc66d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:27.588637 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1384/2772] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:46:27 2022] DELETE /identity/v3/projects/37f34daf6ac94c41b23e1bc899ab671f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:46:27.593495 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8bab6-5bde-4ed2-a0c3-7736418af86f None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:27.593787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8bab6-5bde-4ed2-a0c3-7736418af86f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:27.594108 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8bab6-5bde-4ed2-a0c3-7736418af86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:27.617173 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-94a8bab6-5bde-4ed2-a0c3-7736418af86f None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:46:27.617173 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-94a8bab6-5bde-4ed2-a0c3-7736418af86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:46:27.631972 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:46:27.673586 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1389/2773] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:46:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:46:27.682287 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc204f-69ad-4daf-bdbb-36589cd5af5e None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:27.682769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc204f-69ad-4daf-bdbb-36589cd5af5e None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:27.682900 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc204f-69ad-4daf-bdbb-36589cd5af5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:27.700769 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-b3bc204f-69ad-4daf-bdbb-36589cd5af5e None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:46:27.701960 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-b3bc204f-69ad-4daf-bdbb-36589cd5af5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:46:27.710141 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1385/2774] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:46:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:46:27.714131 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8bab6-5bde-4ed2-a0c3-7736418af86f None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:27.723847 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8bab6-5bde-4ed2-a0c3-7736418af86f None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:27.729493 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e70370-77fc-4e60-bd4c-33dc2680d1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:27.730132 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e70370-77fc-4e60-bd4c-33dc2680d1b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:27.730412 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e70370-77fc-4e60-bd4c-33dc2680d1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:27.730685 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e70370-77fc-4e60-bd4c-33dc2680d1b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:27.731430 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e70370-77fc-4e60-bd4c-33dc2680d1b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:46:27.731761 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e70370-77fc-4e60-bd4c-33dc2680d1b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:27.733338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e70370-77fc-4e60-bd4c-33dc2680d1b7 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:27.745688 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1390/2775] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:46:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:46:27.749883 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc204f-69ad-4daf-bdbb-36589cd5af5e None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:27.759252 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc204f-69ad-4daf-bdbb-36589cd5af5e None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:27.764976 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52749552-ff6d-4aed-86e1-7a0915daae3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:27.765626 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52749552-ff6d-4aed-86e1-7a0915daae3b admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:27.765907 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52749552-ff6d-4aed-86e1-7a0915daae3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:27.766180 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52749552-ff6d-4aed-86e1-7a0915daae3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:27.766931 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52749552-ff6d-4aed-86e1-7a0915daae3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:27.768841 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52749552-ff6d-4aed-86e1-7a0915daae3b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:27.901351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1386/2776] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:46:27 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:46:27.905517 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e70370-77fc-4e60-bd4c-33dc2680d1b7 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:27.916585 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e70370-77fc-4e60-bd4c-33dc2680d1b7 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:27.972944 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f085e94-a2d1-4c5e-a778-92089c13dd66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:27.973596 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f085e94-a2d1-4c5e-a778-92089c13dd66 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:27.973877 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f085e94-a2d1-4c5e-a778-92089c13dd66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:27.974147 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f085e94-a2d1-4c5e-a778-92089c13dd66 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:27.975527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f085e94-a2d1-4c5e-a778-92089c13dd66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:27.977485 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f085e94-a2d1-4c5e-a778-92089c13dd66 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:28.085996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1391/2777] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:46:27 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:46:28.092543 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52749552-ff6d-4aed-86e1-7a0915daae3b admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:28.097737 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52749552-ff6d-4aed-86e1-7a0915daae3b admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:28.103117 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84db5d-882b-4475-a54f-87b536925ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:28.103773 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84db5d-882b-4475-a54f-87b536925ecf admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:28.104058 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84db5d-882b-4475-a54f-87b536925ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:28.104342 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84db5d-882b-4475-a54f-87b536925ecf admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:28.105075 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84db5d-882b-4475-a54f-87b536925ecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:28.106671 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84db5d-882b-4475-a54f-87b536925ecf admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:28.118060 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1387/2778] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:46:28.122036 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f085e94-a2d1-4c5e-a778-92089c13dd66 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:28.132603 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f085e94-a2d1-4c5e-a778-92089c13dd66 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:28.137150 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437ddc7-3abf-44da-b382-99bb7ca3d453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:28.137793 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437ddc7-3abf-44da-b382-99bb7ca3d453 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:28.138072 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437ddc7-3abf-44da-b382-99bb7ca3d453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:28.138343 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437ddc7-3abf-44da-b382-99bb7ca3d453 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:28.139094 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0437ddc7-3abf-44da-b382-99bb7ca3d453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:28.140672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0437ddc7-3abf-44da-b382-99bb7ca3d453 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:28.151747 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1392/2779] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:46:28.155626 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84db5d-882b-4475-a54f-87b536925ecf admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:28.165510 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84db5d-882b-4475-a54f-87b536925ecf admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:28.170166 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c392662-5097-437e-980d-0e8b414ef5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:28.171072 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c392662-5097-437e-980d-0e8b414ef5f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:28.171357 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c392662-5097-437e-980d-0e8b414ef5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:28.171633 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c392662-5097-437e-980d-0e8b414ef5f4 admin admin] PATH_INFO: `/v3/projects/c27945e2c38d45719954abdfabbf6db4/users/94fe289404a24c4baae3a4fc0b5222d4/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:28.185703 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c392662-5097-437e-980d-0e8b414ef5f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c27945e2c38d45719954abdfabbf6db4, user_id=94fe289404a24c4baae3a4fc0b5222d4, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:28.188378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c392662-5097-437e-980d-0e8b414ef5f4 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:28.356825 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1388/2780] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:46:28 2022] PUT /identity/v3/projects/c27945e2c38d45719954abdfabbf6db4/users/94fe289404a24c4baae3a4fc0b5222d4/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:46:28.368373 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec08181-f5b7-43dc-b696-3731577f295f None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:28.372999 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec08181-f5b7-43dc-b696-3731577f295f None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:28.372999 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec08181-f5b7-43dc-b696-3731577f295f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:28.396096 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-0ec08181-f5b7-43dc-b696-3731577f295f None None] MFA Rules not processed for user `94fe289404a24c4baae3a4fc0b5222d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:46:28.397308 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-0ec08181-f5b7-43dc-b696-3731577f295f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:46:28.462994 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1393/2781] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:46:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:46:28.467485 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c392662-5097-437e-980d-0e8b414ef5f4 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:28.510858 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:46:28.529617 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c392662-5097-437e-980d-0e8b414ef5f4 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:28.534432 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1923467-25a5-41f8-aa3e-6f2bdac7dc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:28.544683 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1923467-25a5-41f8-aa3e-6f2bdac7dc16 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:28.544683 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1923467-25a5-41f8-aa3e-6f2bdac7dc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:28.544683 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1923467-25a5-41f8-aa3e-6f2bdac7dc16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:28.544683 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1923467-25a5-41f8-aa3e-6f2bdac7dc16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:28.544683 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1923467-25a5-41f8-aa3e-6f2bdac7dc16 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:28.654648 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1389/2782] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:46:28 2022] POST /identity/v3/projects => generated 387 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:46:28.659331 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec08181-f5b7-43dc-b696-3731577f295f None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:28.670172 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec08181-f5b7-43dc-b696-3731577f295f None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:28.682996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:46:28.725898 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1c441-2609-4ad2-9faf-c43e3183e3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:28.726496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1c441-2609-4ad2-9faf-c43e3183e3d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:28.726777 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1c441-2609-4ad2-9faf-c43e3183e3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:28.727080 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1c441-2609-4ad2-9faf-c43e3183e3d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:28.727775 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1c441-2609-4ad2-9faf-c43e3183e3d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:28.729701 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c1c441-2609-4ad2-9faf-c43e3183e3d2 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:28.820500 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1394/2783] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:46:28 2022] POST /identity/v3/users => generated 294 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:46:28.825925 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1923467-25a5-41f8-aa3e-6f2bdac7dc16 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:28.838251 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1923467-25a5-41f8-aa3e-6f2bdac7dc16 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:28.843120 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b5e2e9-2d82-442b-9973-02eb3b4248e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:28.843793 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5e2e9-2d82-442b-9973-02eb3b4248e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:28.844076 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5e2e9-2d82-442b-9973-02eb3b4248e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:28.844349 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5e2e9-2d82-442b-9973-02eb3b4248e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:28.845080 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b5e2e9-2d82-442b-9973-02eb3b4248e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:28.846681 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b5e2e9-2d82-442b-9973-02eb3b4248e7 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:28.858477 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1390/2784] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:46:28.862790 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1c441-2609-4ad2-9faf-c43e3183e3d2 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:28.874438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1c441-2609-4ad2-9faf-c43e3183e3d2 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:28.879035 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431f68a7-25ce-4cfc-9173-182b82a9a87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:28.879681 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f68a7-25ce-4cfc-9173-182b82a9a87d admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:28.879963 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f68a7-25ce-4cfc-9173-182b82a9a87d admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:28.880236 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431f68a7-25ce-4cfc-9173-182b82a9a87d admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:28.880952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431f68a7-25ce-4cfc-9173-182b82a9a87d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:28.882536 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431f68a7-25ce-4cfc-9173-182b82a9a87d admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:28.892356 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1395/2785] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:46:28.896385 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b5e2e9-2d82-442b-9973-02eb3b4248e7 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:28.906297 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b5e2e9-2d82-442b-9973-02eb3b4248e7 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:28.910886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea8927-070a-4624-845f-3ea7ac0b8ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:28.911758 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea8927-070a-4624-845f-3ea7ac0b8ee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:28.912041 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea8927-070a-4624-845f-3ea7ac0b8ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:28.912319 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea8927-070a-4624-845f-3ea7ac0b8ee1 admin admin] PATH_INFO: `/v3/projects/1fd6c40ff3634103895fd6ed55be5c68/users/64292cd0ba3b4de5b8f1fb62f7ede25b/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:28.926751 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea8927-070a-4624-845f-3ea7ac0b8ee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fd6c40ff3634103895fd6ed55be5c68, user_id=64292cd0ba3b4de5b8f1fb62f7ede25b, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:28.930006 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea8927-070a-4624-845f-3ea7ac0b8ee1 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:29.007597 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1391/2786] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:46:28 2022] PUT /identity/v3/projects/1fd6c40ff3634103895fd6ed55be5c68/users/64292cd0ba3b4de5b8f1fb62f7ede25b/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:46:29.017888 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09d378-3668-4ded-bcc7-83a3da1ff3f3 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:29.018895 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09d378-3668-4ded-bcc7-83a3da1ff3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:29.018895 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09d378-3668-4ded-bcc7-83a3da1ff3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:29.044513 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-5a09d378-3668-4ded-bcc7-83a3da1ff3f3 None None] MFA Rules not processed for user `64292cd0ba3b4de5b8f1fb62f7ede25b`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:46:29.047472 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5a09d378-3668-4ded-bcc7-83a3da1ff3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:46:29.113019 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1396/2787] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:46:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:46:29.118598 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea8927-070a-4624-845f-3ea7ac0b8ee1 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:29.189819 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea8927-070a-4624-845f-3ea7ac0b8ee1 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:29.194437 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4860b9a-4336-4853-a672-071dd8c9d1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:29.195639 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4860b9a-4336-4853-a672-071dd8c9d1c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:29.195920 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4860b9a-4336-4853-a672-071dd8c9d1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:29.196196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4860b9a-4336-4853-a672-071dd8c9d1c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:29.196886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4860b9a-4336-4853-a672-071dd8c9d1c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:29.198798 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4860b9a-4336-4853-a672-071dd8c9d1c1 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:29.312483 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1392/2788] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:46:29 2022] POST /identity/v3/projects => generated 383 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:46:29.316502 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a09d378-3668-4ded-bcc7-83a3da1ff3f3 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:29.327735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a09d378-3668-4ded-bcc7-83a3da1ff3f3 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:29.387515 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba0c73-adc9-4111-a4bc-e949cefc659c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:29.388182 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba0c73-adc9-4111-a4bc-e949cefc659c admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:29.388470 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba0c73-adc9-4111-a4bc-e949cefc659c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:29.388749 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba0c73-adc9-4111-a4bc-e949cefc659c admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:29.389449 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba0c73-adc9-4111-a4bc-e949cefc659c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:29.391393 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba0c73-adc9-4111-a4bc-e949cefc659c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:29.503015 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1397/2789] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:46:29 2022] POST /identity/v3/users => generated 298 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:46:29.507395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4860b9a-4336-4853-a672-071dd8c9d1c1 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:29.517702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4860b9a-4336-4853-a672-071dd8c9d1c1 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:29.522214 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a961bb6-483d-46bc-b430-ca2ebb8d9bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:29.523411 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a961bb6-483d-46bc-b430-ca2ebb8d9bfd admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:29.523691 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a961bb6-483d-46bc-b430-ca2ebb8d9bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:29.523963 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a961bb6-483d-46bc-b430-ca2ebb8d9bfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:29.524674 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a961bb6-483d-46bc-b430-ca2ebb8d9bfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:29.526258 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a961bb6-483d-46bc-b430-ca2ebb8d9bfd admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:29.536511 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1393/2790] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:46:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:46:29.541265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba0c73-adc9-4111-a4bc-e949cefc659c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:29.551305 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba0c73-adc9-4111-a4bc-e949cefc659c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:29.555958 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7305f-c992-44b5-b17b-eac889784953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:29.556832 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7305f-c992-44b5-b17b-eac889784953 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:29.557115 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7305f-c992-44b5-b17b-eac889784953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:29.557398 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7305f-c992-44b5-b17b-eac889784953 admin admin] PATH_INFO: `/v3/projects/2980cda862a84578979b85fc5c30861a/users/60a66903415845ff84938c04fcf2f441/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:29.569317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:46:29.572762 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7305f-c992-44b5-b17b-eac889784953 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2980cda862a84578979b85fc5c30861a, user_id=60a66903415845ff84938c04fcf2f441, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:29.575488 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7305f-c992-44b5-b17b-eac889784953 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:29.704831 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1398/2791] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:46:29 2022] PUT /identity/v3/projects/2980cda862a84578979b85fc5c30861a/users/60a66903415845ff84938c04fcf2f441/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:46:29.708865 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a961bb6-483d-46bc-b430-ca2ebb8d9bfd admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:29.719767 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a961bb6-483d-46bc-b430-ca2ebb8d9bfd admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:29.774398 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c0813-7336-4341-983d-47c678ec6828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:29.775118 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c0813-7336-4341-983d-47c678ec6828 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:29.775400 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c0813-7336-4341-983d-47c678ec6828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:29.775673 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c0813-7336-4341-983d-47c678ec6828 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:29.776386 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47c0813-7336-4341-983d-47c678ec6828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:29.777977 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47c0813-7336-4341-983d-47c678ec6828 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:29.784693 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:46:29.788738 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1394/2792] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:46:29 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:46:29.792888 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7305f-c992-44b5-b17b-eac889784953 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:29.803179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7305f-c992-44b5-b17b-eac889784953 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:29.807740 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a36bf-0ccd-4891-92fb-1c4b3c2b41b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:29.808616 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a36bf-0ccd-4891-92fb-1c4b3c2b41b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:29.808896 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a36bf-0ccd-4891-92fb-1c4b3c2b41b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:29.809169 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a36bf-0ccd-4891-92fb-1c4b3c2b41b8 admin admin] PATH_INFO: `/v3/projects/2980cda862a84578979b85fc5c30861a/users/60a66903415845ff84938c04fcf2f441/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:29.811220 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a36bf-0ccd-4891-92fb-1c4b3c2b41b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2980cda862a84578979b85fc5c30861a, user_id=60a66903415845ff84938c04fcf2f441, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:29.813847 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a36bf-0ccd-4891-92fb-1c4b3c2b41b8 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:29.824515 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1399/2793] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:46:29 2022] PUT /identity/v3/projects/2980cda862a84578979b85fc5c30861a/users/60a66903415845ff84938c04fcf2f441/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:46:29.838184 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11d4c1-8544-4121-aae2-19f244891fbb None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:29.838676 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11d4c1-8544-4121-aae2-19f244891fbb None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:29.838796 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e11d4c1-8544-4121-aae2-19f244891fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:29.864938 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-0e11d4c1-8544-4121-aae2-19f244891fbb None None] MFA Rules not processed for user `60a66903415845ff84938c04fcf2f441`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:46:29.866146 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-0e11d4c1-8544-4121-aae2-19f244891fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:46:29.931053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1395/2794] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:46:29 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:46:29.941577 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a36bf-0ccd-4891-92fb-1c4b3c2b41b8 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:29.943265 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e80a36bf-0ccd-4891-92fb-1c4b3c2b41b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:46:29.999773 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a36bf-0ccd-4891-92fb-1c4b3c2b41b8 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:30.003356 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933a2f7-eea6-41a3-b6d5-654bdef74dc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:30.003991 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933a2f7-eea6-41a3-b6d5-654bdef74dc0 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:30.004274 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933a2f7-eea6-41a3-b6d5-654bdef74dc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:30.004550 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933a2f7-eea6-41a3-b6d5-654bdef74dc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:30.013876 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933a2f7-eea6-41a3-b6d5-654bdef74dc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:30.015334 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933a2f7-eea6-41a3-b6d5-654bdef74dc0 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:30.024503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1400/2795] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:46:29 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:46:30.993228 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e11d4c1-8544-4121-aae2-19f244891fbb None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:30.994638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-0e11d4c1-8544-4121-aae2-19f244891fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:46:31.006327 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e11d4c1-8544-4121-aae2-19f244891fbb None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:31.009831 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af5054-bd8e-464e-81e0-d62a586eb89b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:31.010467 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af5054-bd8e-464e-81e0-d62a586eb89b service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:31.010749 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af5054-bd8e-464e-81e0-d62a586eb89b service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:31.011057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af5054-bd8e-464e-81e0-d62a586eb89b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:31.065662 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af5054-bd8e-464e-81e0-d62a586eb89b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:31.067130 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af5054-bd8e-464e-81e0-d62a586eb89b service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:31.076559 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1396/2796] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:46:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:46:35.913624 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933a2f7-eea6-41a3-b6d5-654bdef74dc0 service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:35.922574 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933a2f7-eea6-41a3-b6d5-654bdef74dc0 service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:35.926427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521dc80d-ddf5-4304-9e91-5f1513d5a72e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:35.927097 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521dc80d-ddf5-4304-9e91-5f1513d5a72e service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:35.927379 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521dc80d-ddf5-4304-9e91-5f1513d5a72e service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:35.927653 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521dc80d-ddf5-4304-9e91-5f1513d5a72e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:35.984101 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521dc80d-ddf5-4304-9e91-5f1513d5a72e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:35.985539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521dc80d-ddf5-4304-9e91-5f1513d5a72e service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:35.994888 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1401/2797] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:46:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:46:40.091015 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af5054-bd8e-464e-81e0-d62a586eb89b service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:40.151109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af5054-bd8e-464e-81e0-d62a586eb89b service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:40.156440 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924e64b-7102-4274-a6db-1d26f078cb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:40.157298 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924e64b-7102-4274-a6db-1d26f078cb11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:40.157578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924e64b-7102-4274-a6db-1d26f078cb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:40.157849 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924e64b-7102-4274-a6db-1d26f078cb11 admin admin] PATH_INFO: `/v3/users/94fe289404a24c4baae3a4fc0b5222d4` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:40.159053 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a924e64b-7102-4274-a6db-1d26f078cb11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94fe289404a24c4baae3a4fc0b5222d4)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:40.160963 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a924e64b-7102-4274-a6db-1d26f078cb11 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:40.253663 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:46:40.374975 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a924e64b-7102-4274-a6db-1d26f078cb11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94fe289404a24c4baae3a4fc0b5222d4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:40.435805 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a924e64b-7102-4274-a6db-1d26f078cb11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94fe289404a24c4baae3a4fc0b5222d4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:40.436417 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a924e64b-7102-4274-a6db-1d26f078cb11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94fe289404a24c4baae3a4fc0b5222d4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:40.443429 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a924e64b-7102-4274-a6db-1d26f078cb11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94fe289404a24c4baae3a4fc0b5222d4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:40.444061 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a924e64b-7102-4274-a6db-1d26f078cb11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94fe289404a24c4baae3a4fc0b5222d4'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:40.461830 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1397/2798] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:46:40 2022] DELETE /identity/v3/users/94fe289404a24c4baae3a4fc0b5222d4 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:46:40.466286 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92586d-87dc-43ac-800b-79e94ff78651 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:40.466778 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92586d-87dc-43ac-800b-79e94ff78651 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:40.466917 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92586d-87dc-43ac-800b-79e94ff78651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:40.483438 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-4a92586d-87dc-43ac-800b-79e94ff78651 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:46:40.484621 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4a92586d-87dc-43ac-800b-79e94ff78651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:46:40.542717 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1402/2799] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:46:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:46:40.552390 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924e64b-7102-4274-a6db-1d26f078cb11 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:40.553801 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a924e64b-7102-4274-a6db-1d26f078cb11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:46:40.610720 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924e64b-7102-4274-a6db-1d26f078cb11 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:40.614314 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020db88f-d156-45cc-846c-1e642091758e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:40.614991 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020db88f-d156-45cc-846c-1e642091758e service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:40.615273 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020db88f-d156-45cc-846c-1e642091758e service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:40.615546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020db88f-d156-45cc-846c-1e642091758e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:40.625076 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020db88f-d156-45cc-846c-1e642091758e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:40.626452 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020db88f-d156-45cc-846c-1e642091758e service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:40.635755 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1398/2800] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:46:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:46:41.043193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92586d-87dc-43ac-800b-79e94ff78651 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:41.044876 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4a92586d-87dc-43ac-800b-79e94ff78651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:46:41.058659 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92586d-87dc-43ac-800b-79e94ff78651 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:41.063370 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f3659-4522-40bb-8a4f-fc501afa47ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:41.064297 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f3659-4522-40bb-8a4f-fc501afa47ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:41.064577 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f3659-4522-40bb-8a4f-fc501afa47ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:41.064910 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f3659-4522-40bb-8a4f-fc501afa47ef admin admin] PATH_INFO: `/v3/projects/c27945e2c38d45719954abdfabbf6db4` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:41.066095 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f3659-4522-40bb-8a4f-fc501afa47ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=c27945e2c38d45719954abdfabbf6db4)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:41.068025 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f3659-4522-40bb-8a4f-fc501afa47ef admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:41.085983 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:46:41.173177 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-616f3659-4522-40bb-8a4f-fc501afa47ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c27945e2c38d45719954abdfabbf6db4', 'request_id': 'req-616f3659-4522-40bb-8a4f-fc501afa47ef'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:41.258995 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-616f3659-4522-40bb-8a4f-fc501afa47ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c27945e2c38d45719954abdfabbf6db4', 'request_id': 'req-616f3659-4522-40bb-8a4f-fc501afa47ef'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:41.265237 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-616f3659-4522-40bb-8a4f-fc501afa47ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c27945e2c38d45719954abdfabbf6db4', 'request_id': 'req-616f3659-4522-40bb-8a4f-fc501afa47ef'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:41.276309 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-616f3659-4522-40bb-8a4f-fc501afa47ef admin admin] The token cache is being invalidate because project c27945e2c38d45719954abdfabbf6db4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:46:41.276622 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-616f3659-4522-40bb-8a4f-fc501afa47ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:41.277240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-616f3659-4522-40bb-8a4f-fc501afa47ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:41.278772 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1403/2801] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:46:41 2022] DELETE /identity/v3/projects/c27945e2c38d45719954abdfabbf6db4 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:46:41.282701 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020db88f-d156-45cc-846c-1e642091758e service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:41.292305 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020db88f-d156-45cc-846c-1e642091758e service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:41.347588 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f82839-f918-4ce4-9fde-d54be45c552b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:41.348580 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f82839-f918-4ce4-9fde-d54be45c552b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:41.348870 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f82839-f918-4ce4-9fde-d54be45c552b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:41.349179 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f82839-f918-4ce4-9fde-d54be45c552b admin admin] PATH_INFO: `/v3/users/64292cd0ba3b4de5b8f1fb62f7ede25b` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:41.350389 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f82839-f918-4ce4-9fde-d54be45c552b admin admin] RBAC: Authorizing `identity:delete_user(user_id=64292cd0ba3b4de5b8f1fb62f7ede25b)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:41.352354 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f82839-f918-4ce4-9fde-d54be45c552b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:41.587403 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e6f82839-f918-4ce4-9fde-d54be45c552b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64292cd0ba3b4de5b8f1fb62f7ede25b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:41.589087 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:46:41.653638 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e6f82839-f918-4ce4-9fde-d54be45c552b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64292cd0ba3b4de5b8f1fb62f7ede25b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:41.654298 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e6f82839-f918-4ce4-9fde-d54be45c552b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64292cd0ba3b4de5b8f1fb62f7ede25b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:41.662012 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e6f82839-f918-4ce4-9fde-d54be45c552b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64292cd0ba3b4de5b8f1fb62f7ede25b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:41.662766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-e6f82839-f918-4ce4-9fde-d54be45c552b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64292cd0ba3b4de5b8f1fb62f7ede25b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:41.680457 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1399/2802] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:46:41 2022] DELETE /identity/v3/users/64292cd0ba3b4de5b8f1fb62f7ede25b => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:46:42.071957 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f3659-4522-40bb-8a4f-fc501afa47ef admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:42.073456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-616f3659-4522-40bb-8a4f-fc501afa47ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:46:42.088487 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f3659-4522-40bb-8a4f-fc501afa47ef admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:42.144840 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada284d1-3b10-41b2-bc99-01a6e75d111e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:42.145631 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada284d1-3b10-41b2-bc99-01a6e75d111e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:42.145916 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada284d1-3b10-41b2-bc99-01a6e75d111e admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:42.146196 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada284d1-3b10-41b2-bc99-01a6e75d111e admin admin] PATH_INFO: `/v3/projects/1fd6c40ff3634103895fd6ed55be5c68` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:42.147399 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada284d1-3b10-41b2-bc99-01a6e75d111e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fd6c40ff3634103895fd6ed55be5c68)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:42.149332 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada284d1-3b10-41b2-bc99-01a6e75d111e admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:42.248752 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ada284d1-3b10-41b2-bc99-01a6e75d111e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fd6c40ff3634103895fd6ed55be5c68', 'request_id': 'req-ada284d1-3b10-41b2-bc99-01a6e75d111e'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:42.332851 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ada284d1-3b10-41b2-bc99-01a6e75d111e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fd6c40ff3634103895fd6ed55be5c68', 'request_id': 'req-ada284d1-3b10-41b2-bc99-01a6e75d111e'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:42.338717 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ada284d1-3b10-41b2-bc99-01a6e75d111e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fd6c40ff3634103895fd6ed55be5c68', 'request_id': 'req-ada284d1-3b10-41b2-bc99-01a6e75d111e'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:42.350693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ada284d1-3b10-41b2-bc99-01a6e75d111e admin admin] The token cache is being invalidate because project 1fd6c40ff3634103895fd6ed55be5c68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:46:42.351024 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ada284d1-3b10-41b2-bc99-01a6e75d111e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:42.351644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-ada284d1-3b10-41b2-bc99-01a6e75d111e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:42.353180 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1404/2803] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:46:42 2022] DELETE /identity/v3/projects/1fd6c40ff3634103895fd6ed55be5c68 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:46:42.357663 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f82839-f918-4ce4-9fde-d54be45c552b admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:42.359088 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-e6f82839-f918-4ce4-9fde-d54be45c552b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:46:42.372598 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f82839-f918-4ce4-9fde-d54be45c552b admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:42.426515 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7e4be5-9b9c-43ed-9d2d-19bf2939e21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:42.427348 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e4be5-9b9c-43ed-9d2d-19bf2939e21f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:42.427629 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e4be5-9b9c-43ed-9d2d-19bf2939e21f admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:42.427901 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7e4be5-9b9c-43ed-9d2d-19bf2939e21f admin admin] PATH_INFO: `/v3/users/60a66903415845ff84938c04fcf2f441` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:42.429073 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e4be5-9b9c-43ed-9d2d-19bf2939e21f admin admin] RBAC: Authorizing `identity:delete_user(user_id=60a66903415845ff84938c04fcf2f441)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:42.431007 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7e4be5-9b9c-43ed-9d2d-19bf2939e21f admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:42.620378 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-fb7e4be5-9b9c-43ed-9d2d-19bf2939e21f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60a66903415845ff84938c04fcf2f441'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:42.681625 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-fb7e4be5-9b9c-43ed-9d2d-19bf2939e21f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60a66903415845ff84938c04fcf2f441'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:42.682194 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-fb7e4be5-9b9c-43ed-9d2d-19bf2939e21f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60a66903415845ff84938c04fcf2f441'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:42.689947 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-fb7e4be5-9b9c-43ed-9d2d-19bf2939e21f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60a66903415845ff84938c04fcf2f441'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:42.695267 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-fb7e4be5-9b9c-43ed-9d2d-19bf2939e21f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60a66903415845ff84938c04fcf2f441'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:42.708274 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1400/2804] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:46:42 2022] DELETE /identity/v3/users/60a66903415845ff84938c04fcf2f441 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:46:43.061208 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada284d1-3b10-41b2-bc99-01a6e75d111e admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:43.062621 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ada284d1-3b10-41b2-bc99-01a6e75d111e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:46:43.076398 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada284d1-3b10-41b2-bc99-01a6e75d111e admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:43.109874 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:46:43.135202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f2f96f-f09d-485e-b424-afd59c012a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:43.136565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2f96f-f09d-485e-b424-afd59c012a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:43.136864 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2f96f-f09d-485e-b424-afd59c012a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:43.137134 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f2f96f-f09d-485e-b424-afd59c012a8c admin admin] PATH_INFO: `/v3/projects/2980cda862a84578979b85fc5c30861a` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:43.138302 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f2f96f-f09d-485e-b424-afd59c012a8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2980cda862a84578979b85fc5c30861a)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:43.140240 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f2f96f-f09d-485e-b424-afd59c012a8c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:43.253644 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-27f2f96f-f09d-485e-b424-afd59c012a8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2980cda862a84578979b85fc5c30861a', 'request_id': 'req-27f2f96f-f09d-485e-b424-afd59c012a8c'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:43.321673 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-27f2f96f-f09d-485e-b424-afd59c012a8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2980cda862a84578979b85fc5c30861a', 'request_id': 'req-27f2f96f-f09d-485e-b424-afd59c012a8c'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:43.327693 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-27f2f96f-f09d-485e-b424-afd59c012a8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2980cda862a84578979b85fc5c30861a', 'request_id': 'req-27f2f96f-f09d-485e-b424-afd59c012a8c'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:43.338733 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-27f2f96f-f09d-485e-b424-afd59c012a8c admin admin] The token cache is being invalidate because project 2980cda862a84578979b85fc5c30861a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:46:43.339044 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-27f2f96f-f09d-485e-b424-afd59c012a8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:43.340293 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-27f2f96f-f09d-485e-b424-afd59c012a8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:46:43.341873 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1405/2805] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:46:43 2022] DELETE /identity/v3/projects/2980cda862a84578979b85fc5c30861a => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:46:43.347884 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d84df6-0f92-4f5f-b2a9-4e4efd6db1cf None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:43.348174 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d84df6-0f92-4f5f-b2a9-4e4efd6db1cf None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:43.348451 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d84df6-0f92-4f5f-b2a9-4e4efd6db1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:43.366267 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-50d84df6-0f92-4f5f-b2a9-4e4efd6db1cf None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:46:43.367511 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-50d84df6-0f92-4f5f-b2a9-4e4efd6db1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:46:43.406685 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:46:43.428373 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1401/2806] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:46:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:46:43.436895 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e338daf-8019-4240-a0fe-d2e0f31a3c14 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:43.437170 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e338daf-8019-4240-a0fe-d2e0f31a3c14 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:43.437452 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e338daf-8019-4240-a0fe-d2e0f31a3c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:43.456184 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-8e338daf-8019-4240-a0fe-d2e0f31a3c14 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:46:43.457384 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8e338daf-8019-4240-a0fe-d2e0f31a3c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:46:43.465719 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1406/2807] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:46:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:46:43.469844 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d84df6-0f92-4f5f-b2a9-4e4efd6db1cf None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:43.479500 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d84df6-0f92-4f5f-b2a9-4e4efd6db1cf None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:43.484056 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d7072-48a6-49a9-9f91-87c37c7e68a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:43.484704 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d7072-48a6-49a9-9f91-87c37c7e68a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:43.484988 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d7072-48a6-49a9-9f91-87c37c7e68a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:43.485265 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d7072-48a6-49a9-9f91-87c37c7e68a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:43.485991 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d7072-48a6-49a9-9f91-87c37c7e68a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:46:43.486322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d7072-48a6-49a9-9f91-87c37c7e68a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:43.487953 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d7072-48a6-49a9-9f91-87c37c7e68a6 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:43.500092 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1402/2808] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:46:43.504624 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e338daf-8019-4240-a0fe-d2e0f31a3c14 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:43.513901 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e338daf-8019-4240-a0fe-d2e0f31a3c14 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:43.518646 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f4319-3ba7-4128-9c82-8e84a2909c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:43.519320 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f4319-3ba7-4128-9c82-8e84a2909c46 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:43.519601 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f4319-3ba7-4128-9c82-8e84a2909c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:43.519876 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f4319-3ba7-4128-9c82-8e84a2909c46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:43.520572 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f4319-3ba7-4128-9c82-8e84a2909c46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:43.522468 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031f4319-3ba7-4128-9c82-8e84a2909c46 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:43.654909 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1407/2809] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:46:43 2022] POST /identity/v3/projects => generated 375 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:46:43.659186 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d7072-48a6-49a9-9f91-87c37c7e68a6 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:43.669192 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d7072-48a6-49a9-9f91-87c37c7e68a6 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:43.724045 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c4ccf9-f4f0-406a-a086-73936a07528e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:43.724702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4ccf9-f4f0-406a-a086-73936a07528e admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:43.724984 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4ccf9-f4f0-406a-a086-73936a07528e admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:43.725255 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4ccf9-f4f0-406a-a086-73936a07528e admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:43.725932 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4ccf9-f4f0-406a-a086-73936a07528e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:43.727887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4ccf9-f4f0-406a-a086-73936a07528e admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:43.801499 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1403/2810] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:46:43 2022] POST /identity/v3/users => generated 288 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:46:43.808730 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f4319-3ba7-4128-9c82-8e84a2909c46 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:43.816708 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f4319-3ba7-4128-9c82-8e84a2909c46 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:43.821260 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4de11f-d6a8-4c6f-befc-4e85088d01b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:43.821911 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4de11f-d6a8-4c6f-befc-4e85088d01b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:43.822195 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4de11f-d6a8-4c6f-befc-4e85088d01b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:43.822472 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4de11f-d6a8-4c6f-befc-4e85088d01b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:43.823223 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4de11f-d6a8-4c6f-befc-4e85088d01b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:43.824811 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4de11f-d6a8-4c6f-befc-4e85088d01b5 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:43.835387 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1408/2811] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:46:43.839217 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c4ccf9-f4f0-406a-a086-73936a07528e admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:43.848995 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c4ccf9-f4f0-406a-a086-73936a07528e admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:43.853544 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb2ab1-88e8-47ec-ad2c-d56f90c36021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:43.854188 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb2ab1-88e8-47ec-ad2c-d56f90c36021 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:43.854468 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb2ab1-88e8-47ec-ad2c-d56f90c36021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:43.855337 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb2ab1-88e8-47ec-ad2c-d56f90c36021 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:43.856057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb2ab1-88e8-47ec-ad2c-d56f90c36021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:43.857640 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb2ab1-88e8-47ec-ad2c-d56f90c36021 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:43.869812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1404/2812] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:46:43.875773 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4de11f-d6a8-4c6f-befc-4e85088d01b5 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:43.887567 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4de11f-d6a8-4c6f-befc-4e85088d01b5 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:43.892951 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc1730-1f69-4123-8e2e-cfeb9b5b9ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:43.893838 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc1730-1f69-4123-8e2e-cfeb9b5b9ffd admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:43.894121 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc1730-1f69-4123-8e2e-cfeb9b5b9ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:43.894398 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc1730-1f69-4123-8e2e-cfeb9b5b9ffd admin admin] PATH_INFO: `/v3/projects/b3fa9aa70861452ab7019403b2e8d866/users/0124f8b43bb047ebb3c6a43ee1c643fd/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:43.908831 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc1730-1f69-4123-8e2e-cfeb9b5b9ffd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3fa9aa70861452ab7019403b2e8d866, user_id=0124f8b43bb047ebb3c6a43ee1c643fd, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:43.911506 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc1730-1f69-4123-8e2e-cfeb9b5b9ffd admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:43.978922 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1409/2813] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:46:43 2022] PUT /identity/v3/projects/b3fa9aa70861452ab7019403b2e8d866/users/0124f8b43bb047ebb3c6a43ee1c643fd/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:46:43.990855 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07b4e7-f558-42fc-8c31-956d95899a01 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:43.993087 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07b4e7-f558-42fc-8c31-956d95899a01 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:43.993087 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07b4e7-f558-42fc-8c31-956d95899a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:44.017944 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-9e07b4e7-f558-42fc-8c31-956d95899a01 None None] MFA Rules not processed for user `0124f8b43bb047ebb3c6a43ee1c643fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:46:44.019192 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-9e07b4e7-f558-42fc-8c31-956d95899a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:46:44.080946 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1405/2814] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:46:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:46:44.093211 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc1730-1f69-4123-8e2e-cfeb9b5b9ffd admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:46:44.094916 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-24fc1730-1f69-4123-8e2e-cfeb9b5b9ffd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:46:44.149850 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc1730-1f69-4123-8e2e-cfeb9b5b9ffd admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:46:44.153953 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c60e7b-4c04-4b72-92ba-db76e1d10779 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:46:44.154586 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c60e7b-4c04-4b72-92ba-db76e1d10779 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:46:44.154889 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c60e7b-4c04-4b72-92ba-db76e1d10779 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:46:44.155163 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c60e7b-4c04-4b72-92ba-db76e1d10779 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:46:44.163323 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c60e7b-4c04-4b72-92ba-db76e1d10779 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:46:44.164748 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c60e7b-4c04-4b72-92ba-db76e1d10779 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:46:44.173844 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1410/2815] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:46:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:46:54.077680 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=79,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:46:54.169909 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:47:07.190182 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07b4e7-f558-42fc-8c31-956d95899a01 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:07.293000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07b4e7-f558-42fc-8c31-956d95899a01 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:07.297558 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396253ee-f545-43d5-a25a-4cf0708dbb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:07.298422 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396253ee-f545-43d5-a25a-4cf0708dbb59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:07.298704 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396253ee-f545-43d5-a25a-4cf0708dbb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:07.299010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396253ee-f545-43d5-a25a-4cf0708dbb59 admin admin] PATH_INFO: `/v3/users/0124f8b43bb047ebb3c6a43ee1c643fd` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:07.300218 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396253ee-f545-43d5-a25a-4cf0708dbb59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0124f8b43bb047ebb3c6a43ee1c643fd)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:07.302133 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396253ee-f545-43d5-a25a-4cf0708dbb59 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:07.511739 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-396253ee-f545-43d5-a25a-4cf0708dbb59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0124f8b43bb047ebb3c6a43ee1c643fd'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:07.579090 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-396253ee-f545-43d5-a25a-4cf0708dbb59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0124f8b43bb047ebb3c6a43ee1c643fd'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:07.579736 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-396253ee-f545-43d5-a25a-4cf0708dbb59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0124f8b43bb047ebb3c6a43ee1c643fd'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:07.587655 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-396253ee-f545-43d5-a25a-4cf0708dbb59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0124f8b43bb047ebb3c6a43ee1c643fd'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:07.588352 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-396253ee-f545-43d5-a25a-4cf0708dbb59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0124f8b43bb047ebb3c6a43ee1c643fd'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:07.605555 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1406/2816] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:47:07 2022] DELETE /identity/v3/users/0124f8b43bb047ebb3c6a43ee1c643fd => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:47:07.610207 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160e7be-ee9f-44fd-bd8b-79eeb07477ee None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:07.610734 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160e7be-ee9f-44fd-bd8b-79eeb07477ee None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:07.610879 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6160e7be-ee9f-44fd-bd8b-79eeb07477ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:07.657301 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-6160e7be-ee9f-44fd-bd8b-79eeb07477ee None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:47:07.658483 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6160e7be-ee9f-44fd-bd8b-79eeb07477ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:07.716028 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1411/2817] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:47:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:47:07.725634 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396253ee-f545-43d5-a25a-4cf0708dbb59 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:07.727062 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-396253ee-f545-43d5-a25a-4cf0708dbb59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:07.768839 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:47:07.820299 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396253ee-f545-43d5-a25a-4cf0708dbb59 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:07.823960 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bbd8a5-759a-45b8-a403-338f5b8b6926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:07.824914 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbd8a5-759a-45b8-a403-338f5b8b6926 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:07.825387 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbd8a5-759a-45b8-a403-338f5b8b6926 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:07.826026 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbd8a5-759a-45b8-a403-338f5b8b6926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:07.835691 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bbd8a5-759a-45b8-a403-338f5b8b6926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:07.837145 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bbd8a5-759a-45b8-a403-338f5b8b6926 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:07.847129 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1407/2818] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:47:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:47:08.500210 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6160e7be-ee9f-44fd-bd8b-79eeb07477ee None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:08.501632 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-6160e7be-ee9f-44fd-bd8b-79eeb07477ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:08.514369 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6160e7be-ee9f-44fd-bd8b-79eeb07477ee None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:08.518963 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c32b7-4672-4123-a9ff-055bcc9d1b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:08.519892 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c32b7-4672-4123-a9ff-055bcc9d1b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:08.520174 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c32b7-4672-4123-a9ff-055bcc9d1b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:08.520474 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c32b7-4672-4123-a9ff-055bcc9d1b35 admin admin] PATH_INFO: `/v3/projects/b3fa9aa70861452ab7019403b2e8d866` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:08.521658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613c32b7-4672-4123-a9ff-055bcc9d1b35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3fa9aa70861452ab7019403b2e8d866)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:08.523672 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613c32b7-4672-4123-a9ff-055bcc9d1b35 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:08.682460 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-613c32b7-4672-4123-a9ff-055bcc9d1b35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3fa9aa70861452ab7019403b2e8d866', 'request_id': 'req-613c32b7-4672-4123-a9ff-055bcc9d1b35'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:08.766870 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-613c32b7-4672-4123-a9ff-055bcc9d1b35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3fa9aa70861452ab7019403b2e8d866', 'request_id': 'req-613c32b7-4672-4123-a9ff-055bcc9d1b35'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:08.772658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-613c32b7-4672-4123-a9ff-055bcc9d1b35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3fa9aa70861452ab7019403b2e8d866', 'request_id': 'req-613c32b7-4672-4123-a9ff-055bcc9d1b35'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:08.784519 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-613c32b7-4672-4123-a9ff-055bcc9d1b35 admin admin] The token cache is being invalidate because project b3fa9aa70861452ab7019403b2e8d866 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80481) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:47:08.787068 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-613c32b7-4672-4123-a9ff-055bcc9d1b35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:08.787068 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-613c32b7-4672-4123-a9ff-055bcc9d1b35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:08.787068 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1412/2819] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:47:08 2022] DELETE /identity/v3/projects/b3fa9aa70861452ab7019403b2e8d866 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:47:08.791878 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee3e69-4012-440b-9919-f5a8757c5b8c None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:08.792349 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee3e69-4012-440b-9919-f5a8757c5b8c None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:08.792450 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee3e69-4012-440b-9919-f5a8757c5b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:08.839915 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-86ee3e69-4012-440b-9919-f5a8757c5b8c None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:47:08.841138 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-86ee3e69-4012-440b-9919-f5a8757c5b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:08.898607 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1408/2820] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:47:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:47:08.906306 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f1a4f-21e0-4c71-846d-289a16d588be None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:08.906631 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f1a4f-21e0-4c71-846d-289a16d588be None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:08.906970 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f1a4f-21e0-4c71-846d-289a16d588be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:08.924817 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-f94f1a4f-21e0-4c71-846d-289a16d588be None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:47:08.926010 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-f94f1a4f-21e0-4c71-846d-289a16d588be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:08.933896 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1413/2821] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:47:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:47:08.938487 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee3e69-4012-440b-9919-f5a8757c5b8c None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:08.948173 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee3e69-4012-440b-9919-f5a8757c5b8c None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:08.953282 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17317151-0062-4092-a01c-e976ff1c68bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:08.953930 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17317151-0062-4092-a01c-e976ff1c68bd admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:08.954212 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17317151-0062-4092-a01c-e976ff1c68bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:08.954488 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17317151-0062-4092-a01c-e976ff1c68bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:08.955278 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17317151-0062-4092-a01c-e976ff1c68bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80482) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:47:08.955586 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17317151-0062-4092-a01c-e976ff1c68bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:08.957186 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17317151-0062-4092-a01c-e976ff1c68bd admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:08.968709 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1409/2822] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:47:08.972941 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f1a4f-21e0-4c71-846d-289a16d588be None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:08.982090 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f1a4f-21e0-4c71-846d-289a16d588be None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:08.986735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb97b2-eb4e-4082-84f8-115a5c6e6a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:08.987407 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb97b2-eb4e-4082-84f8-115a5c6e6a32 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:08.987691 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb97b2-eb4e-4082-84f8-115a5c6e6a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:08.987969 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb97b2-eb4e-4082-84f8-115a5c6e6a32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:08.988670 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb97b2-eb4e-4082-84f8-115a5c6e6a32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:08.990568 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb97b2-eb4e-4082-84f8-115a5c6e6a32 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:09.090026 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1414/2823] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:47:08 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:47:09.093186 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17317151-0062-4092-a01c-e976ff1c68bd admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:09.103350 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17317151-0062-4092-a01c-e976ff1c68bd admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:09.157352 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69ae50-466b-45f5-a8ae-cb686df03285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:09.158008 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69ae50-466b-45f5-a8ae-cb686df03285 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:09.158292 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69ae50-466b-45f5-a8ae-cb686df03285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:09.158570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69ae50-466b-45f5-a8ae-cb686df03285 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:09.159290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69ae50-466b-45f5-a8ae-cb686df03285 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:09.161211 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69ae50-466b-45f5-a8ae-cb686df03285 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:09.345018 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:47:09.350916 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1410/2824] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:47:09 2022] POST /identity/v3/users => generated 288 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:47:09.354938 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb97b2-eb4e-4082-84f8-115a5c6e6a32 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:09.365366 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb97b2-eb4e-4082-84f8-115a5c6e6a32 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:09.369854 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8ddb2-e38a-4e86-a86b-2bf346e2eba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:09.370494 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8ddb2-e38a-4e86-a86b-2bf346e2eba5 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:09.370775 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8ddb2-e38a-4e86-a86b-2bf346e2eba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:09.371079 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8ddb2-e38a-4e86-a86b-2bf346e2eba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:09.371801 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8ddb2-e38a-4e86-a86b-2bf346e2eba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:09.373979 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8ddb2-e38a-4e86-a86b-2bf346e2eba5 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:09.384179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1415/2825] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:47:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:47:09.388000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69ae50-466b-45f5-a8ae-cb686df03285 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:09.397878 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69ae50-466b-45f5-a8ae-cb686df03285 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:09.402489 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0367e179-6791-4182-a83d-42370e436a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:09.403179 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367e179-6791-4182-a83d-42370e436a59 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:09.403455 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367e179-6791-4182-a83d-42370e436a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:09.403729 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0367e179-6791-4182-a83d-42370e436a59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:09.404441 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0367e179-6791-4182-a83d-42370e436a59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:09.406062 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0367e179-6791-4182-a83d-42370e436a59 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:09.416682 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1411/2826] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:47:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:47:09.421039 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8ddb2-e38a-4e86-a86b-2bf346e2eba5 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:09.430765 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8ddb2-e38a-4e86-a86b-2bf346e2eba5 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:09.497720 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac63e3-edaa-4f5f-a0d4-589b29a90528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:09.497720 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac63e3-edaa-4f5f-a0d4-589b29a90528 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:09.497720 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac63e3-edaa-4f5f-a0d4-589b29a90528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:09.497720 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac63e3-edaa-4f5f-a0d4-589b29a90528 admin admin] PATH_INFO: `/v3/projects/fa83dcf81a50486ebe44a68b26393f30/users/1403200557954bb48dc5ab1624165e51/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:09.498144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac63e3-edaa-4f5f-a0d4-589b29a90528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa83dcf81a50486ebe44a68b26393f30, user_id=1403200557954bb48dc5ab1624165e51, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:09.498144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac63e3-edaa-4f5f-a0d4-589b29a90528 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:09.498144 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:47:09.591836 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1416/2827] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:47:09 2022] PUT /identity/v3/projects/fa83dcf81a50486ebe44a68b26393f30/users/1403200557954bb48dc5ab1624165e51/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:47:09.602187 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386e8f9-f5f6-4b99-8692-4de8e8d5b7ea None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:09.602413 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386e8f9-f5f6-4b99-8692-4de8e8d5b7ea None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:09.602687 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386e8f9-f5f6-4b99-8692-4de8e8d5b7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:09.628563 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-5386e8f9-f5f6-4b99-8692-4de8e8d5b7ea None None] MFA Rules not processed for user `1403200557954bb48dc5ab1624165e51`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:47:09.629760 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-5386e8f9-f5f6-4b99-8692-4de8e8d5b7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:09.692137 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1412/2828] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:47:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:47:09.696582 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac63e3-edaa-4f5f-a0d4-589b29a90528 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:09.756831 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac63e3-edaa-4f5f-a0d4-589b29a90528 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:09.761365 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118e2b7-42d9-4a46-8cfe-8661f544d8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:09.762012 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118e2b7-42d9-4a46-8cfe-8661f544d8e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:09.762293 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118e2b7-42d9-4a46-8cfe-8661f544d8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:09.762565 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4118e2b7-42d9-4a46-8cfe-8661f544d8e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:09.763875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4118e2b7-42d9-4a46-8cfe-8661f544d8e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:09.765749 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4118e2b7-42d9-4a46-8cfe-8661f544d8e4 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:09.848247 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1417/2829] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:47:09 2022] POST /identity/v3/projects => generated 375 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:47:09.852050 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5386e8f9-f5f6-4b99-8692-4de8e8d5b7ea None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:09.868355 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5386e8f9-f5f6-4b99-8692-4de8e8d5b7ea None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:09.918151 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71d414-cc7e-46cc-b154-7e0c774c2553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:09.918900 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71d414-cc7e-46cc-b154-7e0c774c2553 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:09.919720 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71d414-cc7e-46cc-b154-7e0c774c2553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:09.919995 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71d414-cc7e-46cc-b154-7e0c774c2553 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:09.920705 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71d414-cc7e-46cc-b154-7e0c774c2553 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:09.922663 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71d414-cc7e-46cc-b154-7e0c774c2553 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:09.995657 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1413/2830] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:47:09 2022] POST /identity/v3/users => generated 288 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:47:09.999587 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118e2b7-42d9-4a46-8cfe-8661f544d8e4 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:10.009875 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4118e2b7-42d9-4a46-8cfe-8661f544d8e4 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:10.014971 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f224f7-c902-4b5c-a728-cea4d1588764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:10.015647 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f224f7-c902-4b5c-a728-cea4d1588764 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:10.015929 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f224f7-c902-4b5c-a728-cea4d1588764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:10.016204 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f224f7-c902-4b5c-a728-cea4d1588764 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:10.016926 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f224f7-c902-4b5c-a728-cea4d1588764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:10.018503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f224f7-c902-4b5c-a728-cea4d1588764 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:10.029503 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1418/2831] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:47:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:47:10.033283 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71d414-cc7e-46cc-b154-7e0c774c2553 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:10.043285 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71d414-cc7e-46cc-b154-7e0c774c2553 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:10.047901 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377babb6-1024-4e55-8c13-fd0d4c143270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:10.048555 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377babb6-1024-4e55-8c13-fd0d4c143270 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:10.048838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377babb6-1024-4e55-8c13-fd0d4c143270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:10.049116 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377babb6-1024-4e55-8c13-fd0d4c143270 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:10.049834 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377babb6-1024-4e55-8c13-fd0d4c143270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:10.051467 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377babb6-1024-4e55-8c13-fd0d4c143270 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:10.061307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1414/2832] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:47:10.065170 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f224f7-c902-4b5c-a728-cea4d1588764 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:10.075051 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f224f7-c902-4b5c-a728-cea4d1588764 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:10.080700 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a75b7-9e9c-4efe-b7d5-ddf54f19cf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:10.081583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a75b7-9e9c-4efe-b7d5-ddf54f19cf9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:10.081862 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a75b7-9e9c-4efe-b7d5-ddf54f19cf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:10.082136 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96a75b7-9e9c-4efe-b7d5-ddf54f19cf9a admin admin] PATH_INFO: `/v3/projects/aff316c8af604850a4837e208775ecd8/users/640fa41fe72540e99812415ae28a7853/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:10.097593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96a75b7-9e9c-4efe-b7d5-ddf54f19cf9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=aff316c8af604850a4837e208775ecd8, user_id=640fa41fe72540e99812415ae28a7853, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:10.100277 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96a75b7-9e9c-4efe-b7d5-ddf54f19cf9a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:10.149899 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1419/2833] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:47:10 2022] PUT /identity/v3/projects/aff316c8af604850a4837e208775ecd8/users/640fa41fe72540e99812415ae28a7853/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:47:10.210368 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f47de2-88c8-4ef5-a79f-f35ba4945b1d None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:10.210646 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f47de2-88c8-4ef5-a79f-f35ba4945b1d None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:10.210949 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f47de2-88c8-4ef5-a79f-f35ba4945b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:10.238197 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-21f47de2-88c8-4ef5-a79f-f35ba4945b1d None None] MFA Rules not processed for user `640fa41fe72540e99812415ae28a7853`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:47:10.239424 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-21f47de2-88c8-4ef5-a79f-f35ba4945b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:10.281362 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:47:10.301806 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1415/2834] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:47:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:47:10.306308 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a75b7-9e9c-4efe-b7d5-ddf54f19cf9a admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:10.367469 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96a75b7-9e9c-4efe-b7d5-ddf54f19cf9a admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:10.372656 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb7048-d043-4391-8da8-715041013951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:10.373309 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb7048-d043-4391-8da8-715041013951 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:10.373589 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb7048-d043-4391-8da8-715041013951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:10.373861 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb7048-d043-4391-8da8-715041013951 admin admin] PATH_INFO: `/v3/projects` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:10.374542 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb7048-d043-4391-8da8-715041013951 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:10.376497 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb7048-d043-4391-8da8-715041013951 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:10.487669 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1420/2835] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:47:10 2022] POST /identity/v3/projects => generated 377 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:47:10.491600 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f47de2-88c8-4ef5-a79f-f35ba4945b1d None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:10.502894 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f47de2-88c8-4ef5-a79f-f35ba4945b1d None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:10.556927 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec36e0-52e0-4204-b193-98c07b005f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:10.557591 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec36e0-52e0-4204-b193-98c07b005f51 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:10.557873 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec36e0-52e0-4204-b193-98c07b005f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:10.558147 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec36e0-52e0-4204-b193-98c07b005f51 admin admin] PATH_INFO: `/v3/users` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:10.558822 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ec36e0-52e0-4204-b193-98c07b005f51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:10.561282 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ec36e0-52e0-4204-b193-98c07b005f51 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:10.653158 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1416/2836] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:47:10 2022] POST /identity/v3/users => generated 295 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:47:10.657219 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb7048-d043-4391-8da8-715041013951 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:10.667726 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb7048-d043-4391-8da8-715041013951 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:10.672778 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb32cff-35f2-483f-808b-fd808a74aa8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:10.673427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb32cff-35f2-483f-808b-fd808a74aa8c admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:10.673710 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb32cff-35f2-483f-808b-fd808a74aa8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:10.674056 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb32cff-35f2-483f-808b-fd808a74aa8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:10.674771 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb32cff-35f2-483f-808b-fd808a74aa8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:10.676398 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb32cff-35f2-483f-808b-fd808a74aa8c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:10.687521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1421/2837] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:47:10.696119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec36e0-52e0-4204-b193-98c07b005f51 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:10.706128 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec36e0-52e0-4204-b193-98c07b005f51 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:10.711026 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d90cd-7ecd-4ac3-adaa-37af72c9ef7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:10.711920 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d90cd-7ecd-4ac3-adaa-37af72c9ef7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:10.712203 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d90cd-7ecd-4ac3-adaa-37af72c9ef7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:10.712477 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d90cd-7ecd-4ac3-adaa-37af72c9ef7d admin admin] PATH_INFO: `/v3/projects/f5a0f5a614d44d0dad6609dfbe29b120/users/2ebb712b18884ae19044192121a13a5b/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:10.726598 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d90cd-7ecd-4ac3-adaa-37af72c9ef7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5a0f5a614d44d0dad6609dfbe29b120, user_id=2ebb712b18884ae19044192121a13a5b, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:10.729287 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d90cd-7ecd-4ac3-adaa-37af72c9ef7d admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:10.796248 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1417/2838] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:47:10 2022] PUT /identity/v3/projects/f5a0f5a614d44d0dad6609dfbe29b120/users/2ebb712b18884ae19044192121a13a5b/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:47:10.800235 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb32cff-35f2-483f-808b-fd808a74aa8c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:10.810665 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb32cff-35f2-483f-808b-fd808a74aa8c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:10.822867 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:47:10.865641 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83757f93-f11b-4502-aef3-aa3a70561600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:10.866298 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83757f93-f11b-4502-aef3-aa3a70561600 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:10.866581 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83757f93-f11b-4502-aef3-aa3a70561600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:10.866877 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83757f93-f11b-4502-aef3-aa3a70561600 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:10.867593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83757f93-f11b-4502-aef3-aa3a70561600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:10.869203 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83757f93-f11b-4502-aef3-aa3a70561600 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:10.879889 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1422/2839] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:47:10.883999 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d90cd-7ecd-4ac3-adaa-37af72c9ef7d admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:10.894255 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d90cd-7ecd-4ac3-adaa-37af72c9ef7d admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:10.898850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba510d-5ad5-408d-a7cc-6b73da577064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:10.899734 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba510d-5ad5-408d-a7cc-6b73da577064 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:10.900013 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba510d-5ad5-408d-a7cc-6b73da577064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:10.900282 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba510d-5ad5-408d-a7cc-6b73da577064 admin admin] PATH_INFO: `/v3/projects/f5a0f5a614d44d0dad6609dfbe29b120/users/2ebb712b18884ae19044192121a13a5b/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:10.902247 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ba510d-5ad5-408d-a7cc-6b73da577064 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5a0f5a614d44d0dad6609dfbe29b120, user_id=2ebb712b18884ae19044192121a13a5b, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:10.904900 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ba510d-5ad5-408d-a7cc-6b73da577064 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:10.914735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1418/2840] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:47:10 2022] PUT /identity/v3/projects/f5a0f5a614d44d0dad6609dfbe29b120/users/2ebb712b18884ae19044192121a13a5b/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:47:10.925050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616a42e-eb9d-42f5-9254-8684c016bc89 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:10.925345 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616a42e-eb9d-42f5-9254-8684c016bc89 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:10.925617 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616a42e-eb9d-42f5-9254-8684c016bc89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:10.951762 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-4616a42e-eb9d-42f5-9254-8684c016bc89 None None] MFA Rules not processed for user `2ebb712b18884ae19044192121a13a5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:47:10.952952 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4616a42e-eb9d-42f5-9254-8684c016bc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:11.018386 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1423/2841] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:47:10 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:47:11.029950 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba510d-5ad5-408d-a7cc-6b73da577064 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:11.031650 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-61ba510d-5ad5-408d-a7cc-6b73da577064 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:11.089474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba510d-5ad5-408d-a7cc-6b73da577064 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:11.093127 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7528f-75e0-48ba-a56e-6aa947d90aa2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:11.093766 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7528f-75e0-48ba-a56e-6aa947d90aa2 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:11.094044 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7528f-75e0-48ba-a56e-6aa947d90aa2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:11.094316 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7528f-75e0-48ba-a56e-6aa947d90aa2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:11.140761 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:47:11.149235 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e7528f-75e0-48ba-a56e-6aa947d90aa2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:11.154073 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e7528f-75e0-48ba-a56e-6aa947d90aa2 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:11.159392 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1419/2842] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:47:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:47:13.171921 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4616a42e-eb9d-42f5-9254-8684c016bc89 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:13.173368 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4616a42e-eb9d-42f5-9254-8684c016bc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:13.185872 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4616a42e-eb9d-42f5-9254-8684c016bc89 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:13.189450 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a656d046-e3b2-4578-9f3d-6294ce3d3029 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:13.190086 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a656d046-e3b2-4578-9f3d-6294ce3d3029 service cinder] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:13.190371 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a656d046-e3b2-4578-9f3d-6294ce3d3029 service cinder] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:13.190645 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a656d046-e3b2-4578-9f3d-6294ce3d3029 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:13.245742 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a656d046-e3b2-4578-9f3d-6294ce3d3029 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:13.247845 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a656d046-e3b2-4578-9f3d-6294ce3d3029 service cinder] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:13.256663 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1424/2843] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:47:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:47:14.964005 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7528f-75e0-48ba-a56e-6aa947d90aa2 service cinder] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:14.973141 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7528f-75e0-48ba-a56e-6aa947d90aa2 service cinder] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:14.978683 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7b8965-def8-4202-8154-90f9f07253e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:14.978683 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b8965-def8-4202-8154-90f9f07253e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:14.978683 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b8965-def8-4202-8154-90f9f07253e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:14.978683 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b8965-def8-4202-8154-90f9f07253e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:14.986649 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7b8965-def8-4202-8154-90f9f07253e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:14.988104 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7b8965-def8-4202-8154-90f9f07253e6 service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:14.997402 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1420/2844] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:47:14 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:47:19.830038 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a656d046-e3b2-4578-9f3d-6294ce3d3029 service cinder] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:19.870793 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=100 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:47:19.891854 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a656d046-e3b2-4578-9f3d-6294ce3d3029 service cinder] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:19.896353 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa1fc7-deaf-477f-8b88-9ad38d84ec90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:19.897215 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa1fc7-deaf-477f-8b88-9ad38d84ec90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:19.897500 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa1fc7-deaf-477f-8b88-9ad38d84ec90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:19.897773 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa1fc7-deaf-477f-8b88-9ad38d84ec90 admin admin] PATH_INFO: `/v3/users/1403200557954bb48dc5ab1624165e51` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:19.898994 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa1fc7-deaf-477f-8b88-9ad38d84ec90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1403200557954bb48dc5ab1624165e51)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:19.900916 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa1fc7-deaf-477f-8b88-9ad38d84ec90 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:20.077789 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-8bfa1fc7-deaf-477f-8b88-9ad38d84ec90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1403200557954bb48dc5ab1624165e51'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:20.162081 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-8bfa1fc7-deaf-477f-8b88-9ad38d84ec90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1403200557954bb48dc5ab1624165e51'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:20.174647 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-8bfa1fc7-deaf-477f-8b88-9ad38d84ec90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1403200557954bb48dc5ab1624165e51'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:20.175337 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-8bfa1fc7-deaf-477f-8b88-9ad38d84ec90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1403200557954bb48dc5ab1624165e51'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:20.183388 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-8bfa1fc7-deaf-477f-8b88-9ad38d84ec90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1403200557954bb48dc5ab1624165e51'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:20.189855 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1425/2845] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:47:19 2022] DELETE /identity/v3/users/1403200557954bb48dc5ab1624165e51 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:47:20.194310 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ee0f1-6ea8-4051-9ad9-9f1ca6425531 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:20.194775 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ee0f1-6ea8-4051-9ad9-9f1ca6425531 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:20.194975 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ee0f1-6ea8-4051-9ad9-9f1ca6425531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:20.212932 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-7a2ee0f1-6ea8-4051-9ad9-9f1ca6425531 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:47:20.214115 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-7a2ee0f1-6ea8-4051-9ad9-9f1ca6425531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:20.271582 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1421/2846] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:47:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:47:20.281226 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa1fc7-deaf-477f-8b88-9ad38d84ec90 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:20.282721 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8bfa1fc7-deaf-477f-8b88-9ad38d84ec90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:20.340023 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa1fc7-deaf-477f-8b88-9ad38d84ec90 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:20.343520 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505af78-be46-4282-846a-c94e254baf7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:20.344149 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505af78-be46-4282-846a-c94e254baf7d service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:20.344429 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505af78-be46-4282-846a-c94e254baf7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:20.344898 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505af78-be46-4282-846a-c94e254baf7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:20.353894 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505af78-be46-4282-846a-c94e254baf7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:20.355426 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505af78-be46-4282-846a-c94e254baf7d service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:20.365381 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1426/2847] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:47:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:47:20.790991 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ee0f1-6ea8-4051-9ad9-9f1ca6425531 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:20.792572 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-7a2ee0f1-6ea8-4051-9ad9-9f1ca6425531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:20.810939 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ee0f1-6ea8-4051-9ad9-9f1ca6425531 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:20.815850 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d363ce6-cc81-4f28-9ead-c5915b9c0a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:20.816857 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d363ce6-cc81-4f28-9ead-c5915b9c0a0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:20.817141 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d363ce6-cc81-4f28-9ead-c5915b9c0a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:20.817418 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d363ce6-cc81-4f28-9ead-c5915b9c0a0b admin admin] PATH_INFO: `/v3/projects/fa83dcf81a50486ebe44a68b26393f30` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:20.818615 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d363ce6-cc81-4f28-9ead-c5915b9c0a0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa83dcf81a50486ebe44a68b26393f30)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:20.820657 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d363ce6-cc81-4f28-9ead-c5915b9c0a0b admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:20.961741 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-3d363ce6-cc81-4f28-9ead-c5915b9c0a0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa83dcf81a50486ebe44a68b26393f30', 'request_id': 'req-3d363ce6-cc81-4f28-9ead-c5915b9c0a0b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:21.063822 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-3d363ce6-cc81-4f28-9ead-c5915b9c0a0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa83dcf81a50486ebe44a68b26393f30', 'request_id': 'req-3d363ce6-cc81-4f28-9ead-c5915b9c0a0b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:21.070265 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-3d363ce6-cc81-4f28-9ead-c5915b9c0a0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa83dcf81a50486ebe44a68b26393f30', 'request_id': 'req-3d363ce6-cc81-4f28-9ead-c5915b9c0a0b'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:21.081417 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-3d363ce6-cc81-4f28-9ead-c5915b9c0a0b admin admin] The token cache is being invalidate because project fa83dcf81a50486ebe44a68b26393f30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:47:21.081704 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-3d363ce6-cc81-4f28-9ead-c5915b9c0a0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:21.082342 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-3d363ce6-cc81-4f28-9ead-c5915b9c0a0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:21.083906 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1422/2848] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:47:20 2022] DELETE /identity/v3/projects/fa83dcf81a50486ebe44a68b26393f30 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:47:21.088052 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505af78-be46-4282-846a-c94e254baf7d service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:21.098456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505af78-be46-4282-846a-c94e254baf7d service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:21.152970 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57116a33-78ab-4ea7-9787-30541835d443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:21.153888 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57116a33-78ab-4ea7-9787-30541835d443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:21.154117 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57116a33-78ab-4ea7-9787-30541835d443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:21.154389 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57116a33-78ab-4ea7-9787-30541835d443 admin admin] PATH_INFO: `/v3/users/640fa41fe72540e99812415ae28a7853` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:21.155592 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57116a33-78ab-4ea7-9787-30541835d443 admin admin] RBAC: Authorizing `identity:delete_user(user_id=640fa41fe72540e99812415ae28a7853)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:21.158143 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57116a33-78ab-4ea7-9787-30541835d443 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:21.159632 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:47:21.374026 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-57116a33-78ab-4ea7-9787-30541835d443 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '640fa41fe72540e99812415ae28a7853'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:21.446697 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-57116a33-78ab-4ea7-9787-30541835d443 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '640fa41fe72540e99812415ae28a7853'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:21.459723 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-57116a33-78ab-4ea7-9787-30541835d443 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '640fa41fe72540e99812415ae28a7853'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:21.460621 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-57116a33-78ab-4ea7-9787-30541835d443 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '640fa41fe72540e99812415ae28a7853'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:21.467606 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-57116a33-78ab-4ea7-9787-30541835d443 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '640fa41fe72540e99812415ae28a7853'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:21.473462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1427/2849] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:47:21 2022] DELETE /identity/v3/users/640fa41fe72540e99812415ae28a7853 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:47:21.939833 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d363ce6-cc81-4f28-9ead-c5915b9c0a0b admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:21.941340 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3d363ce6-cc81-4f28-9ead-c5915b9c0a0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:21.956491 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d363ce6-cc81-4f28-9ead-c5915b9c0a0b admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:22.019915 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc546a-e72b-45e5-b27d-b50c46a6eceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:22.020772 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc546a-e72b-45e5-b27d-b50c46a6eceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:22.021051 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc546a-e72b-45e5-b27d-b50c46a6eceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:22.021322 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc546a-e72b-45e5-b27d-b50c46a6eceb admin admin] PATH_INFO: `/v3/projects/aff316c8af604850a4837e208775ecd8` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:22.022622 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cc546a-e72b-45e5-b27d-b50c46a6eceb admin admin] RBAC: Authorizing `identity:delete_project(project_id=aff316c8af604850a4837e208775ecd8)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:22.024686 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cc546a-e72b-45e5-b27d-b50c46a6eceb admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:22.027165 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:47:22.150152 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c5cc546a-e72b-45e5-b27d-b50c46a6eceb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aff316c8af604850a4837e208775ecd8', 'request_id': 'req-c5cc546a-e72b-45e5-b27d-b50c46a6eceb'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:22.235867 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c5cc546a-e72b-45e5-b27d-b50c46a6eceb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aff316c8af604850a4837e208775ecd8', 'request_id': 'req-c5cc546a-e72b-45e5-b27d-b50c46a6eceb'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:22.242171 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c5cc546a-e72b-45e5-b27d-b50c46a6eceb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aff316c8af604850a4837e208775ecd8', 'request_id': 'req-c5cc546a-e72b-45e5-b27d-b50c46a6eceb'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:22.252800 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c5cc546a-e72b-45e5-b27d-b50c46a6eceb admin admin] The token cache is being invalidate because project aff316c8af604850a4837e208775ecd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:47:22.253090 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c5cc546a-e72b-45e5-b27d-b50c46a6eceb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:22.253712 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-c5cc546a-e72b-45e5-b27d-b50c46a6eceb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:22.255286 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1423/2850] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:47:21 2022] DELETE /identity/v3/projects/aff316c8af604850a4837e208775ecd8 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:47:22.259312 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57116a33-78ab-4ea7-9787-30541835d443 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:22.260751 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-57116a33-78ab-4ea7-9787-30541835d443 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:22.274019 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57116a33-78ab-4ea7-9787-30541835d443 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:22.328657 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1514d88-b295-456a-aeb6-b8ccf691b23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:22.330030 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1514d88-b295-456a-aeb6-b8ccf691b23c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:22.330311 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1514d88-b295-456a-aeb6-b8ccf691b23c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:22.330583 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1514d88-b295-456a-aeb6-b8ccf691b23c admin admin] PATH_INFO: `/v3/users/2ebb712b18884ae19044192121a13a5b` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:22.331799 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1514d88-b295-456a-aeb6-b8ccf691b23c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ebb712b18884ae19044192121a13a5b)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:22.333700 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1514d88-b295-456a-aeb6-b8ccf691b23c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:22.580390 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e1514d88-b295-456a-aeb6-b8ccf691b23c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ebb712b18884ae19044192121a13a5b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:22.657951 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e1514d88-b295-456a-aeb6-b8ccf691b23c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ebb712b18884ae19044192121a13a5b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:22.670251 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e1514d88-b295-456a-aeb6-b8ccf691b23c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ebb712b18884ae19044192121a13a5b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:22.670910 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e1514d88-b295-456a-aeb6-b8ccf691b23c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ebb712b18884ae19044192121a13a5b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:22.678343 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-e1514d88-b295-456a-aeb6-b8ccf691b23c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ebb712b18884ae19044192121a13a5b'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:22.679766 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:47:22.684996 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1428/2851] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:47:22 2022] DELETE /identity/v3/users/2ebb712b18884ae19044192121a13a5b => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:47:23.090706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc546a-e72b-45e5-b27d-b50c46a6eceb admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:23.090706 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-c5cc546a-e72b-45e5-b27d-b50c46a6eceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:23.098903 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc546a-e72b-45e5-b27d-b50c46a6eceb admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:23.155450 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1974a7e-c8c5-4066-be7a-c9084779fa57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:23.156299 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1974a7e-c8c5-4066-be7a-c9084779fa57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:23.157078 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1974a7e-c8c5-4066-be7a-c9084779fa57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:23.157351 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1974a7e-c8c5-4066-be7a-c9084779fa57 admin admin] PATH_INFO: `/v3/projects/f5a0f5a614d44d0dad6609dfbe29b120` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:23.158513 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1974a7e-c8c5-4066-be7a-c9084779fa57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5a0f5a614d44d0dad6609dfbe29b120)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:23.160447 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1974a7e-c8c5-4066-be7a-c9084779fa57 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:23.274783 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a1974a7e-c8c5-4066-be7a-c9084779fa57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5a0f5a614d44d0dad6609dfbe29b120', 'request_id': 'req-a1974a7e-c8c5-4066-be7a-c9084779fa57'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:23.344126 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a1974a7e-c8c5-4066-be7a-c9084779fa57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5a0f5a614d44d0dad6609dfbe29b120', 'request_id': 'req-a1974a7e-c8c5-4066-be7a-c9084779fa57'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:23.351036 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a1974a7e-c8c5-4066-be7a-c9084779fa57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5a0f5a614d44d0dad6609dfbe29b120', 'request_id': 'req-a1974a7e-c8c5-4066-be7a-c9084779fa57'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:23.361502 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a1974a7e-c8c5-4066-be7a-c9084779fa57 admin admin] The token cache is being invalidate because project f5a0f5a614d44d0dad6609dfbe29b120 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:47:23.361791 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a1974a7e-c8c5-4066-be7a-c9084779fa57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:23.362408 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-a1974a7e-c8c5-4066-be7a-c9084779fa57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:47:23.363961 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1424/2852] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:47:23 2022] DELETE /identity/v3/projects/f5a0f5a614d44d0dad6609dfbe29b120 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:47:23.369001 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc679ccc-f2d9-42d4-b958-7fc6a79162a3 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:23.369462 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc679ccc-f2d9-42d4-b958-7fc6a79162a3 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:23.369556 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc679ccc-f2d9-42d4-b958-7fc6a79162a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:23.387589 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-cc679ccc-f2d9-42d4-b958-7fc6a79162a3 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:47:23.388787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-cc679ccc-f2d9-42d4-b958-7fc6a79162a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:23.446577 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1429/2853] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:47:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:47:23.454865 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0de58b-a379-45f0-819d-7eeb6d62dca1 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:23.455168 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0de58b-a379-45f0-819d-7eeb6d62dca1 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:23.455446 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0de58b-a379-45f0-819d-7eeb6d62dca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:23.473853 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-ac0de58b-a379-45f0-819d-7eeb6d62dca1 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:47:23.475095 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-ac0de58b-a379-45f0-819d-7eeb6d62dca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:23.482894 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1425/2854] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:47:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:47:23.487055 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc679ccc-f2d9-42d4-b958-7fc6a79162a3 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:23.496770 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc679ccc-f2d9-42d4-b958-7fc6a79162a3 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:23.501427 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22292cc-a3cc-43c8-97c2-6033377a8139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:23.502074 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22292cc-a3cc-43c8-97c2-6033377a8139 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:23.502352 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22292cc-a3cc-43c8-97c2-6033377a8139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:23.508818 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22292cc-a3cc-43c8-97c2-6033377a8139 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:23.508818 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22292cc-a3cc-43c8-97c2-6033377a8139 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:47:23.508818 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22292cc-a3cc-43c8-97c2-6033377a8139 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:23.508818 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22292cc-a3cc-43c8-97c2-6033377a8139 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:23.517059 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1430/2855] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:47:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:47:23.521344 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0de58b-a379-45f0-819d-7eeb6d62dca1 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:23.530513 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0de58b-a379-45f0-819d-7eeb6d62dca1 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:23.535793 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b4f3d-512f-45b0-afbb-20661fed9714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:23.536433 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b4f3d-512f-45b0-afbb-20661fed9714 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:23.536715 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b4f3d-512f-45b0-afbb-20661fed9714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:23.537015 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b4f3d-512f-45b0-afbb-20661fed9714 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:23.537696 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1b4f3d-512f-45b0-afbb-20661fed9714 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:23.539645 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1b4f3d-512f-45b0-afbb-20661fed9714 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:23.553211 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:47:23.670928 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1426/2856] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:47:23 2022] POST /identity/v3/projects => generated 365 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:47:23.675048 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22292cc-a3cc-43c8-97c2-6033377a8139 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:23.684799 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22292cc-a3cc-43c8-97c2-6033377a8139 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:23.754354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023eee9-6e4c-41d7-95e4-13732e64563a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:23.755087 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023eee9-6e4c-41d7-95e4-13732e64563a admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:23.755373 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023eee9-6e4c-41d7-95e4-13732e64563a admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:23.755651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023eee9-6e4c-41d7-95e4-13732e64563a admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:23.756353 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8023eee9-6e4c-41d7-95e4-13732e64563a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:23.758282 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8023eee9-6e4c-41d7-95e4-13732e64563a admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:23.876155 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1431/2857] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:47:23 2022] POST /identity/v3/users => generated 283 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:47:23.880193 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b4f3d-512f-45b0-afbb-20661fed9714 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:23.890575 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b4f3d-512f-45b0-afbb-20661fed9714 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:23.895613 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b84924-4698-4a6c-91b1-164656309523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:23.896266 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b84924-4698-4a6c-91b1-164656309523 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:23.896552 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b84924-4698-4a6c-91b1-164656309523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:23.896827 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b84924-4698-4a6c-91b1-164656309523 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:23.897594 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b84924-4698-4a6c-91b1-164656309523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:23.899239 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b84924-4698-4a6c-91b1-164656309523 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:23.909403 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1427/2858] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:47:23.913317 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023eee9-6e4c-41d7-95e4-13732e64563a admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:23.923651 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023eee9-6e4c-41d7-95e4-13732e64563a admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:23.928193 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c3f2f-4b8f-4d95-88e3-95c8ad60c491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:23.928841 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c3f2f-4b8f-4d95-88e3-95c8ad60c491 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:23.929124 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c3f2f-4b8f-4d95-88e3-95c8ad60c491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:23.929399 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c3f2f-4b8f-4d95-88e3-95c8ad60c491 admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:23.930117 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c3f2f-4b8f-4d95-88e3-95c8ad60c491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:23.931738 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5c3f2f-4b8f-4d95-88e3-95c8ad60c491 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:23.941986 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1432/2859] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:47:23.946119 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b84924-4698-4a6c-91b1-164656309523 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:23.956518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b84924-4698-4a6c-91b1-164656309523 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:23.961365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89781e-9ca8-471a-b2a1-0268e11c78e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:23.962229 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89781e-9ca8-471a-b2a1-0268e11c78e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:23.962503 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89781e-9ca8-471a-b2a1-0268e11c78e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:23.962773 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89781e-9ca8-471a-b2a1-0268e11c78e3 admin admin] PATH_INFO: `/v3/projects/2be6aab32e34495a8d8c3dfd129a16d0/users/a2714e02f9bb43928e71c24e76fa036d/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:23.976728 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c89781e-9ca8-471a-b2a1-0268e11c78e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2be6aab32e34495a8d8c3dfd129a16d0, user_id=a2714e02f9bb43928e71c24e76fa036d, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:23.980095 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c89781e-9ca8-471a-b2a1-0268e11c78e3 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:24.118509 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1428/2860] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:47:23 2022] PUT /identity/v3/projects/2be6aab32e34495a8d8c3dfd129a16d0/users/a2714e02f9bb43928e71c24e76fa036d/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:47:24.130822 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703fdb4-fadd-47c7-9acb-7885dd9657e5 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:24.156111 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703fdb4-fadd-47c7-9acb-7885dd9657e5 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:24.156111 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703fdb4-fadd-47c7-9acb-7885dd9657e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:24.157852 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-d703fdb4-fadd-47c7-9acb-7885dd9657e5 None None] MFA Rules not processed for user `a2714e02f9bb43928e71c24e76fa036d`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:47:24.159202 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d703fdb4-fadd-47c7-9acb-7885dd9657e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:24.214254 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:47:24.223904 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1433/2861] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:47:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:47:24.235784 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89781e-9ca8-471a-b2a1-0268e11c78e3 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:47:24.237509 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3c89781e-9ca8-471a-b2a1-0268e11c78e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:47:24.293527 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89781e-9ca8-471a-b2a1-0268e11c78e3 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:47:24.297657 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff57bb-b908-48ae-9020-60c12615601e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c81ae2718faf4e59b799add42fb41c4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:47:24.298301 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff57bb-b908-48ae-9020-60c12615601e service cinder] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:47:24.298579 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff57bb-b908-48ae-9020-60c12615601e service cinder] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:47:24.298875 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff57bb-b908-48ae-9020-60c12615601e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:47:24.307156 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff57bb-b908-48ae-9020-60c12615601e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:47:24.308590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff57bb-b908-48ae-9020-60c12615601e service cinder] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:47:24.317724 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1429/2862] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:47:24 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:47:34.298800 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:47:34.313209 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:INSERT=2,keystone:SELECT=43 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:48:02.793027 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703fdb4-fadd-47c7-9acb-7885dd9657e5 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:48:02.854814 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703fdb4-fadd-47c7-9acb-7885dd9657e5 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:48:02.860417 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f193e-b0b4-4f6c-a1bb-841ce3ff5c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:48:02.863946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f193e-b0b4-4f6c-a1bb-841ce3ff5c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:02.863946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f193e-b0b4-4f6c-a1bb-841ce3ff5c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:02.863946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f193e-b0b4-4f6c-a1bb-841ce3ff5c8b admin admin] PATH_INFO: `/v3/users/a2714e02f9bb43928e71c24e76fa036d` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:02.863946 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f193e-b0b4-4f6c-a1bb-841ce3ff5c8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2714e02f9bb43928e71c24e76fa036d)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:48:02.865130 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f193e-b0b4-4f6c-a1bb-841ce3ff5c8b admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:48:02.990741 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4f9f193e-b0b4-4f6c-a1bb-841ce3ff5c8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2714e02f9bb43928e71c24e76fa036d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:48:03.040155 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4f9f193e-b0b4-4f6c-a1bb-841ce3ff5c8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2714e02f9bb43928e71c24e76fa036d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:48:03.053314 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4f9f193e-b0b4-4f6c-a1bb-841ce3ff5c8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2714e02f9bb43928e71c24e76fa036d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:48:03.053968 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4f9f193e-b0b4-4f6c-a1bb-841ce3ff5c8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2714e02f9bb43928e71c24e76fa036d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:48:03.061379 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-4f9f193e-b0b4-4f6c-a1bb-841ce3ff5c8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2714e02f9bb43928e71c24e76fa036d'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:48:03.067578 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1434/2863] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:48:02 2022] DELETE /identity/v3/users/a2714e02f9bb43928e71c24e76fa036d => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:48:03.072066 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c2a32-fcd4-40c9-819c-9d8060d04dd6 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:03.072570 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c2a32-fcd4-40c9-819c-9d8060d04dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:03.072678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904c2a32-fcd4-40c9-819c-9d8060d04dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:03.090199 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-904c2a32-fcd4-40c9-819c-9d8060d04dd6 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:48:03.091425 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-904c2a32-fcd4-40c9-819c-9d8060d04dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:48:03.148135 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1430/2864] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:48:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:48:03.158354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f193e-b0b4-4f6c-a1bb-841ce3ff5c8b admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:48:03.159814 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-4f9f193e-b0b4-4f6c-a1bb-841ce3ff5c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:48:03.217677 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f193e-b0b4-4f6c-a1bb-841ce3ff5c8b admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:48:03.221324 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678fee5-46a0-4cd4-992a-d32e5d00a182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:48:03.221963 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678fee5-46a0-4cd4-992a-d32e5d00a182 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:03.222244 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678fee5-46a0-4cd4-992a-d32e5d00a182 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:03.222518 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678fee5-46a0-4cd4-992a-d32e5d00a182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:03.231420 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678fee5-46a0-4cd4-992a-d32e5d00a182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:48:03.233510 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678fee5-46a0-4cd4-992a-d32e5d00a182 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:48:03.242825 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1435/2865] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:48:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:48:03.636424 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c2a32-fcd4-40c9-819c-9d8060d04dd6 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:48:03.641200 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-904c2a32-fcd4-40c9-819c-9d8060d04dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:48:03.651471 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904c2a32-fcd4-40c9-819c-9d8060d04dd6 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:48:03.656296 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf506282-b227-4f59-8ce7-82bf518ae3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:48:03.657167 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf506282-b227-4f59-8ce7-82bf518ae3e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:03.657454 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf506282-b227-4f59-8ce7-82bf518ae3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:03.657762 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf506282-b227-4f59-8ce7-82bf518ae3e0 admin admin] PATH_INFO: `/v3/projects/2be6aab32e34495a8d8c3dfd129a16d0` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:03.659022 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf506282-b227-4f59-8ce7-82bf518ae3e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2be6aab32e34495a8d8c3dfd129a16d0)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:48:03.660892 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf506282-b227-4f59-8ce7-82bf518ae3e0 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:48:03.777474 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bf506282-b227-4f59-8ce7-82bf518ae3e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2be6aab32e34495a8d8c3dfd129a16d0', 'request_id': 'req-bf506282-b227-4f59-8ce7-82bf518ae3e0'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:48:03.851365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bf506282-b227-4f59-8ce7-82bf518ae3e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2be6aab32e34495a8d8c3dfd129a16d0', 'request_id': 'req-bf506282-b227-4f59-8ce7-82bf518ae3e0'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:48:03.858745 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bf506282-b227-4f59-8ce7-82bf518ae3e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2be6aab32e34495a8d8c3dfd129a16d0', 'request_id': 'req-bf506282-b227-4f59-8ce7-82bf518ae3e0'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:48:03.869292 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bf506282-b227-4f59-8ce7-82bf518ae3e0 admin admin] The token cache is being invalidate because project 2be6aab32e34495a8d8c3dfd129a16d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:48:03.869579 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bf506282-b227-4f59-8ce7-82bf518ae3e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:48:03.870202 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-bf506282-b227-4f59-8ce7-82bf518ae3e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:48:03.871847 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1431/2866] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:48:03 2022] DELETE /identity/v3/projects/2be6aab32e34495a8d8c3dfd129a16d0 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:48:03.876991 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cccd0c-29d0-4f2e-a59a-914702e26df3 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:03.877466 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cccd0c-29d0-4f2e-a59a-914702e26df3 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:03.877622 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cccd0c-29d0-4f2e-a59a-914702e26df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:03.894989 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-d6cccd0c-29d0-4f2e-a59a-914702e26df3 None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:48:03.896198 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d6cccd0c-29d0-4f2e-a59a-914702e26df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:48:03.913288 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:48:03.956191 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1436/2867] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:48:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:48:03.964336 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e6104-429e-4695-ae9c-95e3aada05fb None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:03.964629 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e6104-429e-4695-ae9c-95e3aada05fb None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:03.964901 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e6104-429e-4695-ae9c-95e3aada05fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:03.984094 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-4e6e6104-429e-4695-ae9c-95e3aada05fb None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:48:03.985284 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-4e6e6104-429e-4695-ae9c-95e3aada05fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:48:03.993069 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1432/2868] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:48:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:48:03.997191 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cccd0c-29d0-4f2e-a59a-914702e26df3 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:48:04.006678 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cccd0c-29d0-4f2e-a59a-914702e26df3 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:48:04.011346 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544e00e-88db-49b2-b753-262cedde1ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:48:04.011997 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544e00e-88db-49b2-b753-262cedde1ca5 admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:04.012281 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544e00e-88db-49b2-b753-262cedde1ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:04.012557 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544e00e-88db-49b2-b753-262cedde1ca5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:04.013277 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c544e00e-88db-49b2-b753-262cedde1ca5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80481) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:48:04.013586 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c544e00e-88db-49b2-b753-262cedde1ca5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:48:04.015227 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c544e00e-88db-49b2-b753-262cedde1ca5 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:48:04.026552 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1437/2869] 10.222.0.24 () {58 vars in 1152 bytes} [Fri May 27 03:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:48:04.031056 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e6104-429e-4695-ae9c-95e3aada05fb None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:48:04.039862 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e6104-429e-4695-ae9c-95e3aada05fb None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:48:04.044397 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fbdc2-23d9-4ae5-b8f9-750f785e7e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:48:04.045688 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fbdc2-23d9-4ae5-b8f9-750f785e7e2a admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:04.045973 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fbdc2-23d9-4ae5-b8f9-750f785e7e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:04.046242 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fbdc2-23d9-4ae5-b8f9-750f785e7e2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:04.046970 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8fbdc2-23d9-4ae5-b8f9-750f785e7e2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:48:04.048829 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8fbdc2-23d9-4ae5-b8f9-750f785e7e2a admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:48:04.122099 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1433/2870] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:48:04 2022] POST /identity/v3/projects => generated 379 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:48:04.126210 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544e00e-88db-49b2-b753-262cedde1ca5 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:48:04.136425 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544e00e-88db-49b2-b753-262cedde1ca5 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:48:04.192533 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f57837-6fd7-49db-bbb3-4f36c6f70324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:48:04.193201 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f57837-6fd7-49db-bbb3-4f36c6f70324 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:04.193484 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f57837-6fd7-49db-bbb3-4f36c6f70324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:04.193759 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f57837-6fd7-49db-bbb3-4f36c6f70324 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:04.194453 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f57837-6fd7-49db-bbb3-4f36c6f70324 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:48:04.196443 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f57837-6fd7-49db-bbb3-4f36c6f70324 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:48:04.309085 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1438/2871] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:48:04 2022] POST /identity/v3/users => generated 290 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:48:04.313352 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fbdc2-23d9-4ae5-b8f9-750f785e7e2a admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:48:04.323594 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fbdc2-23d9-4ae5-b8f9-750f785e7e2a admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:48:04.328792 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac3c81-99c3-4bef-9fb9-99ca5d7913e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:48:04.329429 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac3c81-99c3-4bef-9fb9-99ca5d7913e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:04.329748 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac3c81-99c3-4bef-9fb9-99ca5d7913e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:04.329978 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac3c81-99c3-4bef-9fb9-99ca5d7913e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:04.330700 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac3c81-99c3-4bef-9fb9-99ca5d7913e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:48:04.332395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac3c81-99c3-4bef-9fb9-99ca5d7913e5 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:48:04.344458 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1434/2872] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:48:04.348570 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f57837-6fd7-49db-bbb3-4f36c6f70324 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:48:04.358523 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f57837-6fd7-49db-bbb3-4f36c6f70324 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:48:04.363099 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131c6bc-5b60-4e3f-a123-cd87a96c55ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:48:04.363743 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131c6bc-5b60-4e3f-a123-cd87a96c55ed admin admin] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:04.364027 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131c6bc-5b60-4e3f-a123-cd87a96c55ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:04.364301 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131c6bc-5b60-4e3f-a123-cd87a96c55ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:04.365037 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5131c6bc-5b60-4e3f-a123-cd87a96c55ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:48:04.366615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5131c6bc-5b60-4e3f-a123-cd87a96c55ed admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:48:04.376279 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1439/2873] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:48:04 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:48:04.380141 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac3c81-99c3-4bef-9fb9-99ca5d7913e5 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:48:04.390518 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac3c81-99c3-4bef-9fb9-99ca5d7913e5 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:48:04.395155 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c6d9e-5649-459a-8161-06c3d0df79b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:48:04.396023 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c6d9e-5649-459a-8161-06c3d0df79b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:04.396304 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c6d9e-5649-459a-8161-06c3d0df79b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:04.396575 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c6d9e-5649-459a-8161-06c3d0df79b9 admin admin] PATH_INFO: `/v3/projects/696d1db827244c65b179789b47baabe1/users/a98a71f87f4b49fbb98d068065edb1f6/roles/343337c6f25943a9ad412047b70ddefa` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:04.411132 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c6d9e-5649-459a-8161-06c3d0df79b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=696d1db827244c65b179789b47baabe1, user_id=a98a71f87f4b49fbb98d068065edb1f6, role_id=343337c6f25943a9ad412047b70ddefa)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:48:04.413773 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c6d9e-5649-459a-8161-06c3d0df79b9 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:48:04.520571 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1435/2874] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:48:04 2022] PUT /identity/v3/projects/696d1db827244c65b179789b47baabe1/users/a98a71f87f4b49fbb98d068065edb1f6/roles/343337c6f25943a9ad412047b70ddefa => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:48:04.525613 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7682f-7b49-4f40-9620-274850da6e9a None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:04.525906 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7682f-7b49-4f40-9620-274850da6e9a None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:04.526183 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7682f-7b49-4f40-9620-274850da6e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:04.544593 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-55b7682f-7b49-4f40-9620-274850da6e9a None None] MFA Rules not processed for user `93ab1998e69940199adfdba4433a4d08`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:48:04.547487 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-55b7682f-7b49-4f40-9620-274850da6e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:48:04.603385 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1440/2875] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:48:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:48:04.612940 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c6d9e-5649-459a-8161-06c3d0df79b9 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:48:04.614652 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-a57c6d9e-5649-459a-8161-06c3d0df79b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:48:04.637196 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:48:04.674288 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c6d9e-5649-459a-8161-06c3d0df79b9 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:48:04.677926 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c7bc4-5b17-47af-af44-6030abc28bfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:48:04.678564 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c7bc4-5b17-47af-af44-6030abc28bfe service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:04.678868 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c7bc4-5b17-47af-af44-6030abc28bfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:04.679147 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c7bc4-5b17-47af-af44-6030abc28bfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:04.688458 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c7bc4-5b17-47af-af44-6030abc28bfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:48:04.690735 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c7bc4-5b17-47af-af44-6030abc28bfe service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:48:04.699242 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1436/2876] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:48:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:48:09.918526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc04dab-feeb-44e5-9a88-93e956064977 None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:09.919129 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc04dab-feeb-44e5-9a88-93e956064977 None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:09.919129 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc04dab-feeb-44e5-9a88-93e956064977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:09.947933 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-ffc04dab-feeb-44e5-9a88-93e956064977 None None] MFA Rules not processed for user `a98a71f87f4b49fbb98d068065edb1f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:48:09.949165 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-ffc04dab-feeb-44e5-9a88-93e956064977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:48:09.968527 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:48:10.025058 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1441/2877] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:48:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:48:10.029800 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c7bc4-5b17-47af-af44-6030abc28bfe service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:48:10.040561 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c7bc4-5b17-47af-af44-6030abc28bfe service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:48:10.045658 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a9f7c-1596-46ea-803a-c1a6c8ab9897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:48:10.046403 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a9f7c-1596-46ea-803a-c1a6c8ab9897 admin admin] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:10.046753 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a9f7c-1596-46ea-803a-c1a6c8ab9897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:10.047124 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8a9f7c-1596-46ea-803a-c1a6c8ab9897 admin admin] PATH_INFO: `/v3/projects` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:10.047916 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a9f7c-1596-46ea-803a-c1a6c8ab9897 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:48:10.049956 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8a9f7c-1596-46ea-803a-c1a6c8ab9897 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:48:10.133748 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1437/2878] 10.222.0.24 () {60 vars in 1152 bytes} [Fri May 27 03:48:10 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:48:10.140316 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc04dab-feeb-44e5-9a88-93e956064977 None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:48:10.150183 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc04dab-feeb-44e5-9a88-93e956064977 None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:48:10.230909 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe6bfb-0fbd-467d-93e9-6dbbae8c41f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:48:10.230909 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe6bfb-0fbd-467d-93e9-6dbbae8c41f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:10.231204 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe6bfb-0fbd-467d-93e9-6dbbae8c41f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:10.231481 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe6bfb-0fbd-467d-93e9-6dbbae8c41f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:10.232768 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe6bfb-0fbd-467d-93e9-6dbbae8c41f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:48:10.234735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbe6bfb-0fbd-467d-93e9-6dbbae8c41f9 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:48:10.327384 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1442/2879] 10.222.0.24 () {60 vars in 1143 bytes} [Fri May 27 03:48:10 2022] POST /identity/v3/users => generated 296 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:48:10.331161 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a9f7c-1596-46ea-803a-c1a6c8ab9897 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:48:10.342321 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8a9f7c-1596-46ea-803a-c1a6c8ab9897 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:48:10.347067 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c0175-abe9-4590-a2d1-c9a65a6e0034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:48:10.347863 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c0175-abe9-4590-a2d1-c9a65a6e0034 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:10.348150 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c0175-abe9-4590-a2d1-c9a65a6e0034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:10.348447 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c0175-abe9-4590-a2d1-c9a65a6e0034 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:10.349159 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c0175-abe9-4590-a2d1-c9a65a6e0034 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:48:10.350790 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c0175-abe9-4590-a2d1-c9a65a6e0034 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:48:10.365944 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1438/2880] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:48:10.367353 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe6bfb-0fbd-467d-93e9-6dbbae8c41f9 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:48:10.378313 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe6bfb-0fbd-467d-93e9-6dbbae8c41f9 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:48:10.383484 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df212f7f-b009-48d7-869b-d16eacb92358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:48:10.384417 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df212f7f-b009-48d7-869b-d16eacb92358 admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:10.384701 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df212f7f-b009-48d7-869b-d16eacb92358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:10.384978 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df212f7f-b009-48d7-869b-d16eacb92358 admin admin] PATH_INFO: `/v3/projects/71c41060bd5f4028b368df1e232449e2/users/0ca92e72715e4506abcd700faaee8150/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:10.401553 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df212f7f-b009-48d7-869b-d16eacb92358 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71c41060bd5f4028b368df1e232449e2, user_id=0ca92e72715e4506abcd700faaee8150, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:48:10.404320 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df212f7f-b009-48d7-869b-d16eacb92358 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:48:10.504963 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1443/2881] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:48:10 2022] PUT /identity/v3/projects/71c41060bd5f4028b368df1e232449e2/users/0ca92e72715e4506abcd700faaee8150/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:48:10.508859 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c0175-abe9-4590-a2d1-c9a65a6e0034 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:48:10.519334 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c0175-abe9-4590-a2d1-c9a65a6e0034 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:48:10.577128 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8183b4-09b7-44e3-b74e-cb0da886dfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:48:10.578014 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8183b4-09b7-44e3-b74e-cb0da886dfb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:10.578121 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8183b4-09b7-44e3-b74e-cb0da886dfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:10.578348 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8183b4-09b7-44e3-b74e-cb0da886dfb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:10.579125 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8183b4-09b7-44e3-b74e-cb0da886dfb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:48:10.581592 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8183b4-09b7-44e3-b74e-cb0da886dfb1 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:48:10.592936 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1439/2882] 10.222.0.24 () {58 vars in 1121 bytes} [Fri May 27 03:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:48:10.597104 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df212f7f-b009-48d7-869b-d16eacb92358 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:48:10.608498 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df212f7f-b009-48d7-869b-d16eacb92358 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:48:10.613615 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cfaba7-cd77-4b3a-8813-e31d0958957c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:48:10.614634 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfaba7-cd77-4b3a-8813-e31d0958957c admin admin] REQUEST_METHOD: `PUT` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:10.615017 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfaba7-cd77-4b3a-8813-e31d0958957c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:10.615361 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cfaba7-cd77-4b3a-8813-e31d0958957c admin admin] PATH_INFO: `/v3/projects/71c41060bd5f4028b368df1e232449e2/users/0ca92e72715e4506abcd700faaee8150/roles/e5a908bd3ed5445b82a6053246533b88` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:10.617418 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cfaba7-cd77-4b3a-8813-e31d0958957c admin admin] RBAC: Authorizing `identity:create_grant(project_id=71c41060bd5f4028b368df1e232449e2, user_id=0ca92e72715e4506abcd700faaee8150, role_id=e5a908bd3ed5445b82a6053246533b88)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:48:10.620178 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cfaba7-cd77-4b3a-8813-e31d0958957c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:48:10.631347 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1444/2883] 10.222.0.24 () {60 vars in 1482 bytes} [Fri May 27 03:48:10 2022] PUT /identity/v3/projects/71c41060bd5f4028b368df1e232449e2/users/0ca92e72715e4506abcd700faaee8150/roles/e5a908bd3ed5445b82a6053246533b88 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:48:16.051318 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa5239-c555-4f02-bd2a-9cd15d61d653 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:16.051886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa5239-c555-4f02-bd2a-9cd15d61d653 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:16.051886 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa5239-c555-4f02-bd2a-9cd15d61d653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:16.084488 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-96aa5239-c555-4f02-bd2a-9cd15d61d653 None None] MFA Rules not processed for user `0ca92e72715e4506abcd700faaee8150`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:48:16.086298 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-96aa5239-c555-4f02-bd2a-9cd15d61d653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:48:16.179915 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1440/2884] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:48:16 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:48:16.197001 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cfaba7-cd77-4b3a-8813-e31d0958957c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:48:16.198891 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-e9cfaba7-cd77-4b3a-8813-e31d0958957c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:48:16.285411 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:48:16.314424 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cfaba7-cd77-4b3a-8813-e31d0958957c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:48:16.320029 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd1fea-e2f5-4592-a2ff-7fbdf2574834 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:48:16.320681 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd1fea-e2f5-4592-a2ff-7fbdf2574834 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:16.320960 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd1fea-e2f5-4592-a2ff-7fbdf2574834 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:16.321232 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd1fea-e2f5-4592-a2ff-7fbdf2574834 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:16.379702 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd1fea-e2f5-4592-a2ff-7fbdf2574834 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:48:16.381182 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd1fea-e2f5-4592-a2ff-7fbdf2574834 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:48:16.391498 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1445/2885] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:48:16 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:48:26.176578 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:48:26.703670 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=36 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:48:34.599062 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808b5f4-7706-407e-a251-ceff43205f98 None None] REQUEST_METHOD: `POST` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:34.599562 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808b5f4-7706-407e-a251-ceff43205f98 None None] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:34.599679 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808b5f4-7706-407e-a251-ceff43205f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:34.617104 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.auth.core [None req-1808b5f4-7706-407e-a251-ceff43205f98 None None] MFA Rules not processed for user `a98a71f87f4b49fbb98d068065edb1f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80482) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:48:34.618290 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1808b5f4-7706-407e-a251-ceff43205f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:48:34.626208 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1441/2886] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:48:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:48:34.934318 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd1fea-e2f5-4592-a2ff-7fbdf2574834 service nova] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:48:34.936403 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-76bd1fea-e2f5-4592-a2ff-7fbdf2574834 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:48:34.993376 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd1fea-e2f5-4592-a2ff-7fbdf2574834 service nova] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:48:34.997055 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8435331-a103-4218-974a-7ed422d77c65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:48:34.997694 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8435331-a103-4218-974a-7ed422d77c65 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:34.997975 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8435331-a103-4218-974a-7ed422d77c65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:34.998248 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8435331-a103-4218-974a-7ed422d77c65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:35.000198 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b8435331-a103-4218-974a-7ed422d77c65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:48:35.009363 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8435331-a103-4218-974a-7ed422d77c65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:48:35.010787 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8435331-a103-4218-974a-7ed422d77c65 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:48:35.018626 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1446/2887] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:48:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:48:36.717885 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1808b5f4-7706-407e-a251-ceff43205f98 None None] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:48:36.719409 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1808b5f4-7706-407e-a251-ceff43205f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:48:36.816594 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1808b5f4-7706-407e-a251-ceff43205f98 None None] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:48:36.821627 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e167984-e7b5-496e-a832-e42831e1dbb7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f2d0aceef4451a83ee89fac5130a0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:48:36.822268 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e167984-e7b5-496e-a832-e42831e1dbb7 service placement] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:36.822551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e167984-e7b5-496e-a832-e42831e1dbb7 service placement] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:36.822828 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e167984-e7b5-496e-a832-e42831e1dbb7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:36.824603 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-8e167984-e7b5-496e-a832-e42831e1dbb7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:48:36.836644 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e167984-e7b5-496e-a832-e42831e1dbb7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:48:36.838213 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e167984-e7b5-496e-a832-e42831e1dbb7 service placement] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:48:36.848604 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1442/2888] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:48:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:48:38.193539 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8435331-a103-4218-974a-7ed422d77c65 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:48:38.194969 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-b8435331-a103-4218-974a-7ed422d77c65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:48:38.209627 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8435331-a103-4218-974a-7ed422d77c65 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:48:38.214763 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efbb71-3ca5-406d-b3d9-b124e6cbb653 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f2d0aceef4451a83ee89fac5130a0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:48:38.215453 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efbb71-3ca5-406d-b3d9-b124e6cbb653 service placement] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:38.216262 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efbb71-3ca5-406d-b3d9-b124e6cbb653 service placement] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:38.216538 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efbb71-3ca5-406d-b3d9-b124e6cbb653 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:38.218230 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-95efbb71-3ca5-406d-b3d9-b124e6cbb653 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:48:38.229205 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95efbb71-3ca5-406d-b3d9-b124e6cbb653 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:48:38.230905 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95efbb71-3ca5-406d-b3d9-b124e6cbb653 service placement] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:48:38.241230 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1447/2889] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:48:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:48:46.844410 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=44 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:48:48.237047 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=32 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:48:50.424812 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e167984-e7b5-496e-a832-e42831e1dbb7 service placement] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:48:50.437619 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e167984-e7b5-496e-a832-e42831e1dbb7 service placement] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:48:50.442998 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0b243-56ed-443f-ab45-1dad57877c27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:48:50.446551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0b243-56ed-443f-ab45-1dad57877c27 service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:50.446551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0b243-56ed-443f-ab45-1dad57877c27 service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:50.446551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0b243-56ed-443f-ab45-1dad57877c27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:50.446551 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f1e0b243-56ed-443f-ab45-1dad57877c27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:48:50.458662 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e0b243-56ed-443f-ab45-1dad57877c27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:48:50.460523 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e0b243-56ed-443f-ab45-1dad57877c27 service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:48:50.471181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1443/2890] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:48:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:48:53.174338 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efbb71-3ca5-406d-b3d9-b124e6cbb653 service placement] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:48:53.183494 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efbb71-3ca5-406d-b3d9-b124e6cbb653 service placement] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:48:53.187613 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09811aa-469c-4dbb-bd1c-94e3da10f665 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:48:53.188249 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09811aa-469c-4dbb-bd1c-94e3da10f665 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:53.188531 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09811aa-469c-4dbb-bd1c-94e3da10f665 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:53.188806 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09811aa-469c-4dbb-bd1c-94e3da10f665 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:53.197927 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09811aa-469c-4dbb-bd1c-94e3da10f665 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:48:53.199384 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09811aa-469c-4dbb-bd1c-94e3da10f665 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:48:53.209496 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1448/2891] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:48:53 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:48:55.415240 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0b243-56ed-443f-ab45-1dad57877c27 service nova] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:48:55.416702 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-f1e0b243-56ed-443f-ab45-1dad57877c27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:48:55.431132 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0b243-56ed-443f-ab45-1dad57877c27 service nova] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:48:55.436959 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4093a69-4919-4d20-88bd-031d6844c5bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:48:55.437677 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4093a69-4919-4d20-88bd-031d6844c5bb service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:48:55.438032 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4093a69-4919-4d20-88bd-031d6844c5bb service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:48:55.438399 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4093a69-4919-4d20-88bd-031d6844c5bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:48:55.440274 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-d4093a69-4919-4d20-88bd-031d6844c5bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:48:55.452678 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4093a69-4919-4d20-88bd-031d6844c5bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:48:55.454335 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4093a69-4919-4d20-88bd-031d6844c5bb service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:48:55.465590 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1444/2892] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:48:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:49:03.205289 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=6 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:49:05.460824 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=12 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:50:49.668580 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09811aa-469c-4dbb-bd1c-94e3da10f665 service neutron] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:49.677267 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09811aa-469c-4dbb-bd1c-94e3da10f665 service neutron] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:49.680947 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8564ee9f-55e2-450c-b17a-6104a9fa9913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:49.681620 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564ee9f-55e2-450c-b17a-6104a9fa9913 service neutron] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:49.681900 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564ee9f-55e2-450c-b17a-6104a9fa9913 service neutron] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:49.682174 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564ee9f-55e2-450c-b17a-6104a9fa9913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:49.683929 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8564ee9f-55e2-450c-b17a-6104a9fa9913 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:49.697050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8564ee9f-55e2-450c-b17a-6104a9fa9913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:49.698479 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8564ee9f-55e2-450c-b17a-6104a9fa9913 service neutron] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:49.709785 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1449/2893] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:50:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:50:50.883821 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4093a69-4919-4d20-88bd-031d6844c5bb service nova] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:50:50.893624 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4093a69-4919-4d20-88bd-031d6844c5bb service nova] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:50:50.897281 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed73e23-8218-460c-ac70-334f8f87514d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:50:50.897912 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed73e23-8218-460c-ac70-334f8f87514d service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:50.898191 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed73e23-8218-460c-ac70-334f8f87514d service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:50.898466 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed73e23-8218-460c-ac70-334f8f87514d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:50.900261 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-eed73e23-8218-460c-ac70-334f8f87514d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:50.914869 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed73e23-8218-460c-ac70-334f8f87514d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:50:50.916184 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed73e23-8218-460c-ac70-334f8f87514d service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:50:50.927529 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1445/2894] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:50:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:50:54.312060 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dca3e2-195b-4771-9a12-f5762874c7ee None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:50:54.312525 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dca3e2-195b-4771-9a12-f5762874c7ee None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:50:54.312625 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dca3e2-195b-4771-9a12-f5762874c7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:50:54.329420 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-d1dca3e2-195b-4771-9a12-f5762874c7ee None None] MFA Rules not processed for user `a98a71f87f4b49fbb98d068065edb1f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:50:54.330561 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-d1dca3e2-195b-4771-9a12-f5762874c7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:50:54.337594 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1450/2895] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:50:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:51:00.922520 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=6 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:04.324548 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=9 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:51:25.331701 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed73e23-8218-460c-ac70-334f8f87514d service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:51:25.341170 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed73e23-8218-460c-ac70-334f8f87514d service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:51:25.344838 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d122c48-27aa-46b0-a14b-1564d221b698 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:51:25.345505 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d122c48-27aa-46b0-a14b-1564d221b698 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:51:25.345788 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d122c48-27aa-46b0-a14b-1564d221b698 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:51:25.346062 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d122c48-27aa-46b0-a14b-1564d221b698 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:51:25.347952 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-1d122c48-27aa-46b0-a14b-1564d221b698 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:51:25.358729 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d122c48-27aa-46b0-a14b-1564d221b698 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:51:25.360219 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d122c48-27aa-46b0-a14b-1564d221b698 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:51:25.368692 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1446/2896] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:51:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:51:35.365010 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=6 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:52:48.372614 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4cc8c2-de11-424f-b41b-4d57e50031bc None None] REQUEST_METHOD: `POST` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:48.373167 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4cc8c2-de11-424f-b41b-4d57e50031bc None None] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:48.373167 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4cc8c2-de11-424f-b41b-4d57e50031bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:48.391026 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.auth.core [None req-8c4cc8c2-de11-424f-b41b-4d57e50031bc None None] MFA Rules not processed for user `a98a71f87f4b49fbb98d068065edb1f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80481) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:52:48.392204 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-8c4cc8c2-de11-424f-b41b-4d57e50031bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:48.399121 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1451/2897] 10.222.0.24 () {58 vars in 957 bytes} [Fri May 27 03:52:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:52:53.658861 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d122c48-27aa-46b0-a14b-1564d221b698 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:52:53.673752 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d122c48-27aa-46b0-a14b-1564d221b698 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:52:53.680380 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b68a7-513f-4967-aabd-f5d1ee22a64e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:52:53.680641 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b68a7-513f-4967-aabd-f5d1ee22a64e service nova] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:52:53.680921 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b68a7-513f-4967-aabd-f5d1ee22a64e service nova] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:52:53.681373 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b68a7-513f-4967-aabd-f5d1ee22a64e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:52:53.683497 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-3f2b68a7-513f-4967-aabd-f5d1ee22a64e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:52:53.696882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2b68a7-513f-4967-aabd-f5d1ee22a64e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:52:53.698485 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2b68a7-513f-4967-aabd-f5d1ee22a64e service nova] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:52:53.710353 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1447/2898] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:52:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:52:58.386869 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=3 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:53:03.705009 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=6 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:54:28.590568 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4cc8c2-de11-424f-b41b-4d57e50031bc None None] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:54:28.603179 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4cc8c2-de11-424f-b41b-4d57e50031bc None None] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:54:28.608181 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c900c8d-32d4-4ac4-afba-e1604390f6fb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f2d0aceef4451a83ee89fac5130a0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:54:28.608834 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c900c8d-32d4-4ac4-afba-e1604390f6fb service placement] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:28.609119 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c900c8d-32d4-4ac4-afba-e1604390f6fb service placement] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:28.610014 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c900c8d-32d4-4ac4-afba-e1604390f6fb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:28.619263 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c900c8d-32d4-4ac4-afba-e1604390f6fb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:54:28.620856 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c900c8d-32d4-4ac4-afba-e1604390f6fb service placement] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:54:28.631294 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1452/2899] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:54:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:54:35.267647 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b68a7-513f-4967-aabd-f5d1ee22a64e service nova] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:54:35.277067 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b68a7-513f-4967-aabd-f5d1ee22a64e service nova] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:54:35.280692 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f035d-3fac-49ab-9e48-55bfaa3e9541 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:54:35.281325 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f035d-3fac-49ab-9e48-55bfaa3e9541 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:35.281608 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f035d-3fac-49ab-9e48-55bfaa3e9541 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:35.282457 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f035d-3fac-49ab-9e48-55bfaa3e9541 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:35.290694 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f035d-3fac-49ab-9e48-55bfaa3e9541 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:54:35.292198 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f035d-3fac-49ab-9e48-55bfaa3e9541 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:54:35.301181 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1448/2900] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:54:35 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:54:36.186292 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c900c8d-32d4-4ac4-afba-e1604390f6fb service placement] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:54:36.199082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c900c8d-32d4-4ac4-afba-e1604390f6fb service placement] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:54:36.207299 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9cce9-7b9b-4c06-99a9-e4f458bfc039 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a9592143014f3287cd75c60ca17237', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:54:36.208369 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9cce9-7b9b-4c06-99a9-e4f458bfc039 service nova] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:36.208795 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9cce9-7b9b-4c06-99a9-e4f458bfc039 service nova] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:36.209081 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9cce9-7b9b-4c06-99a9-e4f458bfc039 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:36.219790 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9cce9-7b9b-4c06-99a9-e4f458bfc039 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:54:36.221381 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9cce9-7b9b-4c06-99a9-e4f458bfc039 service nova] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:54:36.232456 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1453/2901] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:54:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:54:45.297000 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=6 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:54:46.227664 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=12 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:54:50.744887 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f035d-3fac-49ab-9e48-55bfaa3e9541 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:54:50.753750 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f035d-3fac-49ab-9e48-55bfaa3e9541 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:54:50.757529 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864ea4d-920f-43fe-a33e-cde8e9b4b553 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af0160cd3da4a0bb391b81f083715d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:54:50.758154 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864ea4d-920f-43fe-a33e-cde8e9b4b553 service neutron] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:50.758438 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864ea4d-920f-43fe-a33e-cde8e9b4b553 service neutron] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:50.760307 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864ea4d-920f-43fe-a33e-cde8e9b4b553 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:50.818979 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864ea4d-920f-43fe-a33e-cde8e9b4b553 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:54:50.820444 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864ea4d-920f-43fe-a33e-cde8e9b4b553 service neutron] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:54:50.830712 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1449/2902] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:54:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:54:54.039899 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9cce9-7b9b-4c06-99a9-e4f458bfc039 service nova] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:54:54.051372 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9cce9-7b9b-4c06-99a9-e4f458bfc039 service nova] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:54:54.056454 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7dfc5a-f852-41e7-9c63-9f0763be40a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f2d0aceef4451a83ee89fac5130a0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:54:54.057088 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dfc5a-f852-41e7-9c63-9f0763be40a2 service placement] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:54.057374 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dfc5a-f852-41e7-9c63-9f0763be40a2 service placement] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:54.057647 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7dfc5a-f852-41e7-9c63-9f0763be40a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:54.059521 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-7d7dfc5a-f852-41e7-9c63-9f0763be40a2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:54:54.075883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7dfc5a-f852-41e7-9c63-9f0763be40a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:54:54.075883 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7dfc5a-f852-41e7-9c63-9f0763be40a2 service placement] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:54:54.085643 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1454/2903] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:54:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:54:57.613610 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864ea4d-920f-43fe-a33e-cde8e9b4b553 service neutron] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:54:57.624992 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864ea4d-920f-43fe-a33e-cde8e9b4b553 service neutron] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:54:57.630829 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488c23a2-a3d4-4d1f-86ff-24a1483f1ab0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f2d0aceef4451a83ee89fac5130a0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:54:57.631501 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c23a2-a3d4-4d1f-86ff-24a1483f1ab0 service placement] REQUEST_METHOD: `GET` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:57.631782 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c23a2-a3d4-4d1f-86ff-24a1483f1ab0 service placement] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:57.632101 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c23a2-a3d4-4d1f-86ff-24a1483f1ab0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:57.633882 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-488c23a2-a3d4-4d1f-86ff-24a1483f1ab0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:54:57.645531 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488c23a2-a3d4-4d1f-86ff-24a1483f1ab0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:54:57.647130 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488c23a2-a3d4-4d1f-86ff-24a1483f1ab0 service placement] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:54:57.657546 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1450/2904] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:54:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:54:57.750028 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7dfc5a-f852-41e7-9c63-9f0763be40a2 service placement] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:54:57.760821 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7dfc5a-f852-41e7-9c63-9f0763be40a2 service placement] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:54:57.765658 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486daf45-cad6-4214-abc1-88023b301f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:54:57.766526 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486daf45-cad6-4214-abc1-88023b301f0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:57.766856 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486daf45-cad6-4214-abc1-88023b301f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:57.767132 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486daf45-cad6-4214-abc1-88023b301f0c admin admin] PATH_INFO: `/v3/users/a98a71f87f4b49fbb98d068065edb1f6` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:57.768308 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486daf45-cad6-4214-abc1-88023b301f0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a98a71f87f4b49fbb98d068065edb1f6)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:54:57.770197 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486daf45-cad6-4214-abc1-88023b301f0c admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:54:57.961174 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-486daf45-cad6-4214-abc1-88023b301f0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a98a71f87f4b49fbb98d068065edb1f6'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:54:58.067803 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-486daf45-cad6-4214-abc1-88023b301f0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a98a71f87f4b49fbb98d068065edb1f6'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:54:58.084082 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-486daf45-cad6-4214-abc1-88023b301f0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a98a71f87f4b49fbb98d068065edb1f6'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:54:58.085070 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-486daf45-cad6-4214-abc1-88023b301f0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a98a71f87f4b49fbb98d068065edb1f6'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:54:58.091967 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-486daf45-cad6-4214-abc1-88023b301f0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a98a71f87f4b49fbb98d068065edb1f6'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:54:58.098436 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1455/2905] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:54:57 2022] DELETE /identity/v3/users/a98a71f87f4b49fbb98d068065edb1f6 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:54:58.352685 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488c23a2-a3d4-4d1f-86ff-24a1483f1ab0 service placement] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:54:58.368338 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488c23a2-a3d4-4d1f-86ff-24a1483f1ab0 service placement] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:54:58.425922 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252321da-6543-49d8-b3f1-8ca4e80a6f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:54:58.426781 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252321da-6543-49d8-b3f1-8ca4e80a6f55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:58.427090 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252321da-6543-49d8-b3f1-8ca4e80a6f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:58.427365 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252321da-6543-49d8-b3f1-8ca4e80a6f55 admin admin] PATH_INFO: `/v3/projects/696d1db827244c65b179789b47baabe1` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:58.428538 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252321da-6543-49d8-b3f1-8ca4e80a6f55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=696d1db827244c65b179789b47baabe1)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:54:58.430433 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252321da-6543-49d8-b3f1-8ca4e80a6f55 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:54:58.529925 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-252321da-6543-49d8-b3f1-8ca4e80a6f55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '696d1db827244c65b179789b47baabe1', 'request_id': 'req-252321da-6543-49d8-b3f1-8ca4e80a6f55'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:54:58.611461 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-252321da-6543-49d8-b3f1-8ca4e80a6f55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '696d1db827244c65b179789b47baabe1', 'request_id': 'req-252321da-6543-49d8-b3f1-8ca4e80a6f55'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:54:58.618333 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-252321da-6543-49d8-b3f1-8ca4e80a6f55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '696d1db827244c65b179789b47baabe1', 'request_id': 'req-252321da-6543-49d8-b3f1-8ca4e80a6f55'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:54:58.628646 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-252321da-6543-49d8-b3f1-8ca4e80a6f55 admin admin] The token cache is being invalidate because project 696d1db827244c65b179789b47baabe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:54:58.628934 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-252321da-6543-49d8-b3f1-8ca4e80a6f55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:54:58.629564 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-252321da-6543-49d8-b3f1-8ca4e80a6f55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:54:58.631150 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1451/2906] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:54:58 2022] DELETE /identity/v3/projects/696d1db827244c65b179789b47baabe1 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:54:58.635106 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486daf45-cad6-4214-abc1-88023b301f0c admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:54:58.636493 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-486daf45-cad6-4214-abc1-88023b301f0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:54:58.649284 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486daf45-cad6-4214-abc1-88023b301f0c admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:54:58.705778 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a53faae-c881-4022-9aa4-9b1efe1133f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:54:58.706640 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53faae-c881-4022-9aa4-9b1efe1133f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:58.706948 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53faae-c881-4022-9aa4-9b1efe1133f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:58.707255 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a53faae-c881-4022-9aa4-9b1efe1133f1 admin admin] PATH_INFO: `/v3/users/0ca92e72715e4506abcd700faaee8150` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:58.708416 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a53faae-c881-4022-9aa4-9b1efe1133f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ca92e72715e4506abcd700faaee8150)` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:54:58.710578 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a53faae-c881-4022-9aa4-9b1efe1133f1 admin admin] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:54:58.907044 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1a53faae-c881-4022-9aa4-9b1efe1133f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ca92e72715e4506abcd700faaee8150'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:54:58.996479 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1a53faae-c881-4022-9aa4-9b1efe1133f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ca92e72715e4506abcd700faaee8150'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:54:59.002319 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:54:59.009397 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1a53faae-c881-4022-9aa4-9b1efe1133f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ca92e72715e4506abcd700faaee8150'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:54:59.010050 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1a53faae-c881-4022-9aa4-9b1efe1133f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ca92e72715e4506abcd700faaee8150'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:54:59.017432 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.notifications [None req-1a53faae-c881-4022-9aa4-9b1efe1133f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ca92e72715e4506abcd700faaee8150'} {{(pid=80481) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:54:59.023360 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1456/2907] 10.222.0.24 () {58 vars in 1223 bytes} [Fri May 27 03:54:58 2022] DELETE /identity/v3/users/0ca92e72715e4506abcd700faaee8150 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:54:59.283312 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252321da-6543-49d8-b3f1-8ca4e80a6f55 admin admin] Authenticating user token {{(pid=80482) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:54:59.284837 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.fernet_utils [None req-252321da-6543-49d8-b3f1-8ca4e80a6f55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80482) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:54:59.300376 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252321da-6543-49d8-b3f1-8ca4e80a6f55 admin admin] Validating token access rules against request {{(pid=80482) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:54:59.309856 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:54:59.358395 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd405934-586c-4036-99d9-0b8d60c8a444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '93ab1998e69940199adfdba4433a4d08', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d3e8df9c2aa4b4588906f73830bd805', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80482) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:54:59.359328 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd405934-586c-4036-99d9-0b8d60c8a444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:54:59.359611 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd405934-586c-4036-99d9-0b8d60c8a444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:54:59.359888 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd405934-586c-4036-99d9-0b8d60c8a444 admin admin] PATH_INFO: `/v3/projects/71c41060bd5f4028b368df1e232449e2` {{(pid=80482) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:54:59.361109 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd405934-586c-4036-99d9-0b8d60c8a444 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71c41060bd5f4028b368df1e232449e2)` {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:54:59.363057 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd405934-586c-4036-99d9-0b8d60c8a444 admin admin] RBAC: Authorization granted {{(pid=80482) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:54:59.447878 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-dd405934-586c-4036-99d9-0b8d60c8a444 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71c41060bd5f4028b368df1e232449e2', 'request_id': 'req-dd405934-586c-4036-99d9-0b8d60c8a444'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:54:59.524554 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-dd405934-586c-4036-99d9-0b8d60c8a444 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71c41060bd5f4028b368df1e232449e2', 'request_id': 'req-dd405934-586c-4036-99d9-0b8d60c8a444'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:54:59.530800 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-dd405934-586c-4036-99d9-0b8d60c8a444 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71c41060bd5f4028b368df1e232449e2', 'request_id': 'req-dd405934-586c-4036-99d9-0b8d60c8a444'} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:54:59.542277 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-dd405934-586c-4036-99d9-0b8d60c8a444 admin admin] The token cache is being invalidate because project 71c41060bd5f4028b368df1e232449e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80482) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:54:59.542565 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-dd405934-586c-4036-99d9-0b8d60c8a444 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:54:59.543204 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG keystone.notifications [None req-dd405934-586c-4036-99d9-0b8d60c8a444 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80482) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:54:59.544737 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: [pid: 80482|app: 0|req: 1452/2908] 10.222.0.24 () {58 vars in 1232 bytes} [Fri May 27 03:54:59 2022] DELETE /identity/v3/projects/71c41060bd5f4028b368df1e232449e2 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:55:09.350535 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=9,keystone:DELETE=1 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:55:09.539091 dv-cinder-828805-4-iscsi devstack@keystone.service[80482]: DEBUG dbcounter [-] [80482] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80482) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:59:34.584890 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a53faae-c881-4022-9aa4-9b1efe1133f1 admin admin] Authenticating user token {{(pid=80481) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:59:34.586290 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-1a53faae-c881-4022-9aa4-9b1efe1133f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:34.723797 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a53faae-c881-4022-9aa4-9b1efe1133f1 admin admin] Validating token access rules against request {{(pid=80481) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:59:34.784354 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a003a01-5cab-4bde-9866-1d192326fc51 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95f2d0aceef4451a83ee89fac5130a0c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ceffdd955324aec84ba048d2137ae00', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80481) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:59:34.785002 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a003a01-5cab-4bde-9866-1d192326fc51 service placement] REQUEST_METHOD: `GET` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:59:34.785301 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a003a01-5cab-4bde-9866-1d192326fc51 service placement] SCRIPT_NAME: `/identity` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:59:34.785580 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a003a01-5cab-4bde-9866-1d192326fc51 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80481) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:59:34.787383 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.fernet_utils [None req-5a003a01-5cab-4bde-9866-1d192326fc51 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80481) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:59:34.798735 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a003a01-5cab-4bde-9866-1d192326fc51 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:59:34.800428 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a003a01-5cab-4bde-9866-1d192326fc51 service placement] RBAC: Authorization granted {{(pid=80481) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:59:34.810635 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: [pid: 80481|app: 0|req: 1457/2909] 10.222.0.24 () {60 vars in 1365 bytes} [Fri May 27 03:59:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:59:44.806411 dv-cinder-828805-4-iscsi devstack@keystone.service[80481]: DEBUG dbcounter [-] [80481] Writing DB stats keystone:SELECT=89 {{(pid=80481) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}